Aerial floating image display device, aerial floating image display system, and portable terminal

Abstract

To provide a more suitable aerial floating image display device which, according to the present invention, contributes to Sustainable Development Goals: 3, "Ensure healthy lives and promote well-being for all at all ages," 9, "Build resilient infrastructure, promote inclusive and sustainable industrialization and foster innovation," and 11, "Make cities and human settlements inclusive, safe, and resilient and sustainable."SOLUTION: The aerial floating image display device comprises: a video processing unit; a display unit for displaying the video having undergone video processing by the video processing unit; an optical system for generating an aerial floating video on the basis of the video displayed by the display unit; a detection unit for detecting contact of an object with a display range of the aerial floating video; and a communication unit for communicating with the portable terminal of a user. When contact of the portable terminal with the display range is detected, the aerial floating image display device causes an image of interest designated on the portable terminal side to be displayed as an aerial floating image.SELECTED DRAWING: Figure 16

Description

[Technical field]

[0001] The present invention relates to a floating-in-the-air image display device. [Background technology]

[0002] The airborne information display technology is disclosed in, for example, Patent Document 1. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-128722 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the disclosure of Patent Document 1 does not sufficiently consider configurations for obtaining practical brightness and quality for the floating image, or configurations for allowing users to more enjoyably view the floating image.

[0005] An object of the present invention is to provide a more suitable floating-in-the-air image display device. [Means for solving the problem]

[0006] In order to solve the above problem, for example, the configuration described in the claims is adopted. The present application includes a plurality of means for solving the above problem, and an example thereof may be configured as follows. One embodiment is a floating-in-the-air image display device, which includes an image processing unit, a display unit that displays an image processed by the image processing unit, an optical system that generates a floating-in-the-air image based on the image displayed by the display unit, a detection unit that detects contact of an object with the display range of the floating-in-the-air image, and a communication unit that communicates with a user's mobile terminal, and when contact of the mobile terminal with the display range is detected, a target image specified on the mobile terminal side is displayed as a floating-in-the-air image. Effect of the Invention

[0007] According to the present invention, a more suitable floating-in-the-air image display device can be realized. Other objects, configurations, and effects will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram showing an example of a usage form of a space floating image display device according to an embodiment of the present invention; [Figure 2A] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 2B] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 2C] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 2D] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a floating-in-the-air image display device according to an embodiment of the present invention; [Figure 2E] 1 is a projection diagram of a retroreflector constituting a floating-in-the-air image display device according to an embodiment of the present invention; [Figure 2F] FIG. 2 is a top view of a retroreflector constituting a floating-in-the-air image display device according to an embodiment of the present invention. [Figure 2G] FIG. 2 is a perspective view showing a corner reflector constituting a retroreflector constituting a floating-in-the-air image display device according to an embodiment of the present invention. [Figure 2H] FIG. 2 is a top view showing a corner reflector constituting a retroreflector constituting a floating-in-the-air image display device according to an embodiment of the present invention. [Figure 2I] 1 is a side view showing a corner reflector constituting a retroreflector constituting a floating-in-the-air image display device according to an embodiment of the present invention; FIG. [Diagram 3] 1 is a diagram showing a configuration example of a space floating image display device according to an embodiment of the present invention; [Figure 4A] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4B] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4C] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4D] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4E] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4F] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4G] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4H] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4I] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4J] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4K] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4L] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4M] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4N] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Figure 4O] FIG. 1 is a diagram showing an example of a configuration of a space floating image display device according to an embodiment of the present invention. [Diagram 5] FIG. 2 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. [Figure 6]FIG. 2 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. [Figure 7] FIG. 2 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. [Figure 8] 1 is a layout diagram showing a main part of a space floating image display device according to an embodiment of the present invention; [Figure 9] 1 is a cross-sectional view showing a configuration of a display device according to an embodiment of the present invention. [Figure 10] 1 is a cross-sectional view showing a configuration of a display device according to an embodiment of the present invention. [Figure 11] 1 is an explanatory diagram for explaining a light source diffusion characteristic of an image display device according to an embodiment of the present invention. [Figure 12] 1 is an explanatory diagram for explaining the diffusion characteristics of a video display device according to an embodiment of the present invention; [Figure 13A] 1 is a diagram illustrating an example of a problem to be solved by image processing according to an embodiment of the present invention; [Figure 13B] FIG. 4 is an explanatory diagram of an example of image processing according to an embodiment of the present invention. [Figure 13C] FIG. 4 is an explanatory diagram of an example of a video display process according to an embodiment of the present invention. [Figure 13D] FIG. 4 is an explanatory diagram of an example of a video display process according to an embodiment of the present invention. [Figure 14] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 15] 1 is a diagram showing the configuration of a space floating image display system according to an embodiment of the present invention; [Figure 16] 1 is a diagram showing an overview of a space floating image display system according to an embodiment of the present invention; [Figure 17] FIG. 2 is a diagram showing a configuration of a mobile terminal according to an embodiment of the present invention. [Figure 18A] 1 is a perspective view showing an insertion and touch operation of a mobile terminal into a floating image in space according to an embodiment of the present invention; [Figure 18B] 1 is an XY plan view showing an insertion and touch operation of a mobile terminal into a floating image in space according to an embodiment of the present invention; [Figure 18C]1 is a YZ plane view showing an insertion and touch operation of a mobile terminal into a floating image in space according to an embodiment of the present invention. FIG. [Figure 18D] 1 is an XZ plane view showing an insertion and touch operation of a mobile terminal into a floating image in space according to an embodiment of the present invention; [Figure 19A] 1 is a perspective view showing the posture when inserting and touching a mobile terminal into a floating image in accordance with an embodiment of the present invention; [Figure 19B] 1A to 1C are XY plan views showing examples of postures when inserting and touching a mobile terminal into a floating-in-space image according to an embodiment of the present invention. [Figure 19C] 1A to 1C are YZ plan views showing examples of postures when inserting and touching a mobile terminal into a floating-in-space image according to an embodiment of the present invention. [Figure 19D] 1A to 1C are XZ plane views showing examples of postures when inserting and touching a mobile terminal into a floating-in-space image according to an embodiment of the present invention. [Figure 19E] 1A to 1C are YZ plan views showing examples of postures when inserting and touching a mobile terminal into a floating-in-space image according to an embodiment of the present invention. [Figure 19F] 1A to 1C are YZ plan views showing examples of postures when inserting and touching a mobile terminal into a floating-in-space image according to an embodiment of the present invention. [Figure 20A] 1 is a YZ plane view showing an example of a position when inserting or touching a mobile terminal into a floating-in-space image according to an embodiment of the present invention; [Figure 20B] 1 is an XY plan view showing an example of a position when inserting or touching a mobile terminal into a floating-in-space image according to an embodiment of the present invention; [Figure 20C] 1 is a YZ plane view showing an example of a position when inserting or touching a mobile terminal into a floating-in-space image according to an embodiment of the present invention; [Figure 21A] FIG. 13 is an explanatory diagram showing an example of a guide image being displayed on a floating-in-space image according to an embodiment of the present invention. [Figure 21B] FIG. 13 is an explanatory diagram showing an example of a guide image being displayed on a floating-in-space image according to an embodiment of the present invention. [Figure 22A] 1 is an XY plan view showing an example of a mobile terminal touching a floating image in space according to an embodiment of the present invention; [Figure 22B]1A to 1C are XZ plane views showing examples of invisible areas caused by contact of a mobile terminal with a floating-in-space image according to an embodiment of the present invention. [Figure 22C] FIG. 11 is a YZ plan view showing an example of shading caused by contact of a mobile terminal with a floating-in-space image according to an embodiment of the present invention. [Figure 23A] 1A is an xy plan view showing an example of a guide image displayed on a floating-in-space image according to an embodiment of the present invention, and an example of an invisible area due to contact with a mobile terminal. [Figure 23B] 11 is an xy plan view showing an example of displaying a guide image on a space floating image according to one embodiment of the present invention and an example of limiting a display area. [Figure 24A] FIG. 2 is an xy plan view showing an example of displaying a guide image on a floating-in-space image according to an embodiment of the present invention. [Figure 24B] FIG. 2 is an xy plan view showing an example of displaying a guide image on a floating-in-space image according to an embodiment of the present invention. [Figure 24C] 1A and 1B are explanatory diagrams showing an example of displaying a guide image on a space floating image and an example of limiting a display area according to an embodiment of the present invention. [Figure 24D] FIG. 4 is an explanatory diagram showing an example of a guide voice output according to an embodiment of the present invention. [Figure 25A] 1 is a YZ plane cross-sectional view showing an example of the arrangement of sensors in a space floating image display device according to an embodiment of the present invention. [Figure 25B] FIG. 2 is an xy plan view showing an example of the configuration of an aerial operation detection sensor according to an embodiment of the present invention. [Figure 25C] 1 is a YZ plane cross-sectional view showing an example of the arrangement of sensors in a space floating image display device according to an embodiment of the present invention. [Figure 26A] FIG. 4 is an explanatory diagram showing a display example of an application screen on a mobile terminal according to an embodiment of the present invention. [Figure 26B] FIG. 4 is an explanatory diagram showing a display example of an application screen on a mobile terminal according to an embodiment of the present invention. [Figure 26C] FIG. 4 is an explanatory diagram showing a display example of an application screen on a mobile terminal according to an embodiment of the present invention. [Figure 27A] FIG. 11 is an explanatory diagram showing a display example of a QR code screen on a mobile terminal according to an embodiment of the present invention. [Figure 27B] 1 is a YZ plane cross-sectional view showing an example of capturing an image of a QR code of a mobile terminal in a space floating image display device according to an embodiment of the present invention; [Figure 28A] FIG. 1 is a perspective view showing an example of a QR code displayed in a space floating image according to an embodiment of the present invention. [Figure 28B] 1 is a YZ plane cross-sectional view showing an example of capturing an image of a QR code of a space-floating image of a space-floating image display device by a mobile terminal according to an embodiment of the present invention; [Figure 28C] FIG. 11 is a perspective view showing an example of a QR code displayed on a screen of a second display device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the description of the embodiments, and various changes and modifications can be made by those skilled in the art within the scope of the technical ideas disclosed in this specification. In addition, in all the drawings for explaining the present invention, the same reference numerals are given to parts having the same functions, and repeated explanations thereof may be omitted.

[0010] The following embodiment relates to an image display device capable of transmitting an image produced by image light from an image emission source through a transparent member that divides a space, such as glass, and displaying the image as a floating image outside the transparent member. In the following description of the embodiment, an image floating in space is expressed by the term "floating image in space". Instead of this term, it is also acceptable to express it as "aerial image", "spatial image", "floating image in space", "floating optical image of a displayed image", "floating optical image of a displayed image", etc. The term "floating image in space" that is mainly used in the description of the embodiment is used as a representative example of these terms.

[0011] According to the following embodiment, for example, a suitable image display device can be realized for bank ATMs, station ticket vending machines, digital signage, etc. For example, currently, touch panels are usually used in bank ATMs, station ticket vending machines, etc., but by using a transparent glass surface or a light-transmitting plate material, high-resolution image information can be displayed in a space-floating state on the glass surface or light-transmitting plate material. At this time, by making the divergence angle of the emitted image light small, that is, an acute angle, and further aligning it with a specific polarization, only the normal reflected light is efficiently reflected by the retroreflector, so that the light utilization efficiency is high, and ghost images that occur in addition to the main space-floating image, which was a problem in the conventional retroreflection method, can be suppressed, and a clear space-floating image can be obtained. In addition, a device including the light source of this embodiment can provide a novel and highly usable space-floating image display device (space-floating image display system) that can significantly reduce power consumption. In addition, for example, a space-floating image display device for vehicles that can display a so-called unidirectional space-floating image that can be viewed inside and / or outside the vehicle can be provided. <Example 1>

[0012] <An example of how the space floating image display device can be used> FIG. 1 is a diagram showing an example of a usage form of a space-floating image display device according to an embodiment of the present invention, and is a diagram showing the overall configuration of the space-floating image display device according to this embodiment. The specific configuration of the space-floating image display device will be described in detail using FIG. 2 and the like, but light with a narrow-angle directional characteristic and specific polarization is emitted from the image display device 1 as an image light beam, and is once incident on the retroreflector 2 after reflection in the optical system in the space-floating image display device, and is retroreflected and transmitted through a transparent member 100 (glass, etc.), forming a real aerial image (space-floating image 3) on the outside of the glass surface. In the following embodiment, the retroreflector 2 (retroreflector) is used as an example of the retroreflector. However, the retroreflector 2 of the present invention is not limited to a flat plate, and is used as an example of a concept including a sheet-like retroreflector attached to a flat or non-flat member, and an entire assembly in which a sheet-like retroreflector is attached to a flat or non-flat member. Furthermore, since the light rays reflected by the retroreflector 2 have the optical property of forming an image, the retroreflector 2 may be expressed as an imaging optical member or an imaging optical plate.

[0013] In addition, in a store or the like, a space is divided by a show window (also called "window glass") 105, which is a light-transmitting member such as glass. According to the space floating image display device of this embodiment, it is possible to transmit the floating image through such a transparent member and display it in one direction to the outside and / or inside of the store (space).

[0014] 1, the inside of the window glass 105 (inside the store) is shown in the depth direction, and the outside (for example, the sidewalk) is shown in the foreground. On the other hand, by providing the window glass 105 with a means for reflecting a specific polarized wave, it is possible to form an aerial image at a desired position inside the store.

[0015] <Example of optical system configuration for space floating image display device> Fig. 2A is a diagram showing an example of the configuration of an optical system of a space-floating image display device according to an embodiment of the present invention. The configuration of the space-floating image display device will be described in more detail with reference to Fig. 2A. As shown in Fig. 2A(1), a display device 1 that diverges specific polarized image light at a narrow angle is provided in the oblique direction of a transparent member 100 such as glass. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates specific polarized light having a narrow-angle diffusion characteristic.

[0016] The image light of a specific polarization from the display device 1 is reflected by a polarization separation member 101 (in the figure, the polarization separation member 101 is formed into a sheet shape and adhered to the transparent member 100) having a film that selectively reflects the image light of a specific polarization provided on a transparent member 100, and enters the retroreflector 2. A λ / 4 plate 21 is provided on the image light incidence surface of the retroreflector 2. The image light is made to pass through the λ / 4 plate 21 twice, when it enters the retroreflector 2 and when it leaves, and is polarized and converted from the specific polarization to the other polarization. Here, the polarization separation member 101 that selectively reflects the image light of a specific polarization has a property of transmitting the polarized light of the other polarization that has been polarized and converted, so the image light of the specific polarization after polarization conversion passes through the polarization separation member 101. The image light that has passed through the polarization separation member 101 forms a space floating image 3, which is a real image, outside the transparent member 100. 2A shows an example in which the chief ray of the image light incident on the retroreflector 2 is incident at an angle of 90° to the retroreflector 2. However, the incident angle of the chief ray of the image light on the retroreflector 2 is not limited to 90°, and may be, for example, 90°±15°.

[0017] Here, a first example of the polarization design in the optical system of FIG. 2A will be described. For example, the display device 1 may be configured to emit S-polarized image light to the polarization separation member 101, and the polarization separation member 101 may have the property of reflecting S-polarized light and transmitting P-polarized light. In this case, the S-polarized image light that reaches the polarization separation member 101 from the display device 1 is reflected by the polarization separation member 101 and travels toward the retroreflector 2. When the image light is reflected by the retroreflector 2, it passes through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2 twice, so that the image light is converted from S-polarized light to P-polarized light. The image light converted to P-polarized light travels again toward the polarization separation member 101. Here, the polarization separation member 101 has the property of reflecting S-polarized light and transmitting P-polarized light, so that the P-polarized image light passes through the polarization separation member 101 and then through the transparent member 100. The image light transmitted through the transparent member 100 is generated by the retroreflector 2, and therefore forms a floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 with respect to the polarization separation member 101. This polarization design allows the floating image 3 to be formed appropriately.

[0018] Next, a second example of the polarization design in the optical system of FIG. 2A will be described. For example, the display device 1 may be configured to emit P-polarized image light to the polarization separation member 101, and the polarization separation member 101 may have a property of reflecting P-polarized light and transmitting S-polarized light. In this case, the P-polarized image light that reaches the polarization separation member 101 from the display device 1 is reflected by the polarization separation member 101 and travels toward the retroreflector 2. When the image light is reflected by the retroreflector 2, it passes through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2 twice, so that the image light is converted from P-polarized light to S-polarized light. The image light converted to S-polarized light travels again toward the polarization separation member 101. Here, the polarization separation member 101 has a property of reflecting P-polarized light and transmitting S-polarized light, so that the S-polarized image light passes through the polarization separation member 101 and then through the transparent member 100. The image light transmitted through the transparent member 100 is generated by the retroreflector 2, and therefore forms a floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 with respect to the polarization separation member 101. This polarization design allows the floating image 3 to be formed appropriately.

[0019] The light forming the floating image 3 is a collection of light rays that converge from the retroreflector 2 to the optical image of the floating image 3, and these light rays continue to travel straight even after passing through the optical image of the floating image 3. Therefore, the floating image 3 is an image with high directivity, unlike the diffuse image light formed on a screen by a general projector or the like. Therefore, in the configuration of FIG. 2A, when a user views the floating image 3 from the direction of the arrow A, the floating image 3 is viewed as a bright image. However, when another person views the floating image 3 from the direction of the arrow B, the floating image 3 cannot be viewed as an image at all. This characteristic is very suitable for use in a system that displays images that require high security or highly confidential images that should be kept secret from people facing the user.

[0020] Depending on the performance of the retroreflector 2, the polarization axis of the image light after reflection may become uneven. Also, the reflection angle may become uneven. Such uneven light may not maintain the polarization state and traveling angle assumed in the design. For example, light with such a polarization state and traveling angle that is not assumed in the design may re-enter the image display surface side of the liquid crystal display panel 11 directly from the position of the retroreflector 2 without passing through the polarization separation member. Such light with a polarization state and traveling angle that is not assumed in the design may re-enter the image display surface side of the liquid crystal display panel 11 after being reflected by a component in the space floating image display device. Such light that re-enters the image display surface side of the liquid crystal display panel 11 may be re-reflected by the image display surface of the liquid crystal display panel 11 that constitutes the display device 1, generating a ghost image and possibly degrading the image quality of the space floating image. Therefore, in this embodiment, an absorbing polarizing plate 12 may be provided on the image display surface of the display device 1. The image light emitted from the display device 1 is transmitted through the absorptive polarizing plate 12, and the reflected light returning from the polarization separation member 101 is absorbed by the absorptive polarizing plate 12, thereby suppressing the re-reflection. This makes it possible to prevent degradation of image quality due to ghost images of spatially floating images. Specifically, if the display device 1 is configured to emit S-polarized image light to the polarization separation member 101, the absorptive polarizing plate 12 may be a polarizing plate that absorbs P-polarized light. Also, if the display device 1 is configured to emit P-polarized image light to the polarization separation member 101, the absorptive polarizing plate 12 may be a polarizing plate that absorbs S-polarized light.

[0021] The above-mentioned polarization separation member 101 may be formed, for example, of a reflective polarizing plate or a metal multilayer film that reflects a specific polarized wave.

[0022] 2A(2) shows an example of the surface shape of a retroreflector as a representative retroreflector 2. A ray of light incident on the interior of regularly arranged hexagonal prisms is reflected by the walls and bottoms of the hexagonal prisms and emitted as retroreflected light in a direction corresponding to the incident light, and a real floating image is displayed based on the image displayed on the display device 1.

[0023] The resolution of this floating image in space depends not only on the resolution of the liquid crystal display panel 11, but also on the outer shape D and pitch P of the retroreflector 2 shown in Fig. 2A(2). For example, when using a 7-inch WUXGA (1920 x 1200 pixels) liquid crystal display panel, even if one pixel (one triplet) is about 80 μm, if the diameter D of the retroreflector is 240 μm and the pitch is 300 μm, one pixel of the floating image in space will be equivalent to 300 μm. For this reason, the effective resolution of the floating image in space will be reduced to about 1 / 3.

[0024] Therefore, in order to make the resolution of the spatial floating image equal to that of the display device 1, it is desirable to make the diameter and pitch of the retroreflective portion close to one pixel of the liquid crystal display panel. On the other hand, in order to suppress the occurrence of moire due to the retroreflective plate and the pixels of the liquid crystal display panel, it is preferable to design the pitch ratio of each to be a different integer multiple of one pixel. In addition, it is preferable to arrange the shape so that none of the sides of the retroreflective portion overlaps with any of the sides of one pixel of the liquid crystal display panel.

[0025] The surface shape of the retroreflector according to this embodiment is not limited to the above example. It may have various surface shapes that realize retroreflection. Specifically, a retroreflection element in which triangular pyramid prisms, hexagonal pyramid prisms, other polygonal prisms, or a combination of these are periodically arranged may be provided on the surface of the retroreflector of this embodiment. Alternatively, a retroreflection element in which these prisms are periodically arranged to form a cube corner may be provided on the surface of the retroreflector of this embodiment. These can also be expressed as a corner reflector array or a multifaceted reflector array. Alternatively, a capsule lens type retroreflection element in which glass beads are periodically arranged may be provided on the surface of the retroreflector of this embodiment. The detailed configuration of these retroreflection elements can be achieved by using existing technology, so a detailed description will be omitted. Specifically, the techniques disclosed in Japanese Patent Application Laid-Open Nos. 2001-33609, 2001-264525, 2005-181555, 2008-70898, and 2009-229942 may be used.

[0026] <Another configuration example 1 of the optical system of the space floating image display device> Another example of the configuration of the optical system of the space floating image display device will be described with reference to Fig. 2B. In Fig. 2B, the components with the same reference numerals as Fig. 2A have the same functions and configurations as Fig. 2A. For such components, repeated explanations will be omitted to simplify the explanation.

[0027] In the optical system of FIG. 2B, as in FIG. 2A, image light of a specific polarization is output from the display device 1. The image light of a specific polarization output from the display device 1 is input to the polarization separation member 101B. The polarization separation member 101B is a member that selectively transmits the image light of a specific polarization. Unlike the polarization separation member 101 of FIG. 2A, the polarization separation member 101B is not integrated with the transparent member 100, but has an independent plate-like shape. Therefore, the polarization separation member 101B may be expressed as a polarization separation plate. The polarization separation member 101B may be configured as a reflective polarizing plate configured by attaching a polarization separation sheet to a transparent member, for example. Alternatively, the transparent member may be formed of a metal multilayer film that selectively transmits a specific polarization and reflects a polarization of another specific polarization. In FIG. 2B, the polarization separation member 101B is configured to transmit the image light of a specific polarization output from the display device 1.

[0028] The image light transmitted through the polarization separation member 101B is incident on the retroreflector 2. A λ / 4 plate 21 is provided on the image light incident surface of the retroreflector. The image light is polarized and converted from a specific polarization to the other polarization by passing through the λ / 4 plate 21 twice, when it enters the retroreflector and when it leaves the retroreflector. Here, the polarization separation member 101B has a property of reflecting the polarized light of the other polarization converted by the λ / 4 plate 21, so that the image light after polarization conversion is reflected by the polarization separation member 101B. The image light reflected by the polarization separation member 101B passes through the transparent member 100 and forms a space floating image 3, which is a real image, outside the transparent member 100.

[0029] Here, a first example of the polarization design in the optical system of FIG. 2B will be described. For example, the display device 1 may be configured to emit P-polarized image light to the polarization separation member 101B, and the polarization separation member 101B may have a property of reflecting S-polarized light and transmitting P-polarized light. In this case, the P-polarized image light that reaches the polarization separation member 101B from the display device 1 passes through the polarization separation member 101B and travels toward the retroreflector 2. When the image light is reflected by the retroreflector 2, it passes through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2 twice, so that the image light is converted from P-polarized light to S-polarized light. The image light converted to S-polarized light travels again toward the polarization separation member 101B. Here, the polarization separation member 101B has a property of reflecting S-polarized light and transmitting P-polarized light, so that the S-polarized image light is reflected by the polarization separation member 101 and passes through the transparent member 100. Since the image light transmitted through the transparent member 100 is generated by the retroreflector 2, it forms a space-floating image 3, which is an optical image of the display image of the display device 1, at a position that is in a mirror relationship with the display image of the display device 1 with respect to the polarization separation member 101B. This polarization design allows the space-floating image 3 to be formed appropriately.

[0030] Next, a second example of the polarization design in the optical system of FIG. 2B will be described. For example, the display device 1 may be configured to emit S-polarized image light to the polarization separation member 101B, which may have the property of reflecting P-polarized light and transmitting S-polarized light. In this case, the S-polarized image light reaching the polarization separation member 101B from the display device 1 passes through the polarization separation member 101B and travels toward the retroreflector 2. When the image light is reflected by the retroreflector 2, it passes through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2 twice, so that the image light is converted from S-polarized light to P-polarized light. The image light converted to P-polarized light travels again toward the polarization separation member 101B. Here, the polarization separation member 101B has the property of reflecting P-polarized light and transmitting S-polarized light, so that the P-polarized image light is reflected by the polarization separation member 101 and passes through the transparent member 100. The image light transmitted through the transparent member 100 is generated by the retroreflector 2, and therefore forms a space-floating image 3, which is an optical image of the display image of the display device 1, at a position that is in a mirror relationship with the display image of the display device 1 with respect to the polarization separation member 101B. With this polarization design, the space-floating image 3 can be formed suitably.

[0031] In FIG. 2B, the image display surface of the display device 1 and the surface of the retroreflector 2 are arranged parallel to each other. The polarization separation member 101B is arranged at an angle α (for example, 30°) with respect to the image display surface of the display device 1 and the surface of the retroreflector 2. Then, in the reflection of the polarization separation member 101B, the traveling direction of the image light reflected by the polarization separation member 101B (the direction of the chief ray of the image light) differs from the traveling direction of the image light incident from the retroreflector 2 (the direction of the chief ray of the image light) by an angle β (for example, 60°). By configuring in this way, in the optical system of FIG. 2B, the image light is output at a predetermined angle shown in the figure toward the outside of the transparent member 100, and the space-floating image 3, which is a real image, is formed. In the configuration of FIG. 2B, when a user views the space-floating image 3 from the direction of the arrow A, the space-floating image 3 is viewed as a bright image. However, when another person views the space-floating image 3 from the direction of the arrow B, the space-floating image 3 cannot be viewed as an image at all. This characteristic is extremely suitable for use in a system that displays images that require high security or highly confidential images that should be concealed from people directly facing the user.

[0032] As described above, the optical system of FIG. 2B is an optical system having a different configuration from the optical system of FIG. 2A, but can form a suitable floating-in-space image in the same way as the optical system of FIG. 2A.

[0033] An absorptive polarizing plate may be provided on the surface of the transparent member 100 facing the polarization separation member 101B. The absorptive polarizing plate may transmit the polarized wave of the image light from the polarization separation member 101B and absorb the polarized wave that is 90° out of phase with the polarized wave of the image light from the polarization separation member 101B. In this way, the image light for forming the space-floating image 3 is sufficiently transmitted, while the external light incident from the space-floating image 3 side of the transparent member 100 can be reduced by about 50%. This makes it possible to reduce stray light in the optical system of FIG. 2B due to the external light incident from the space-floating image 3 side of the transparent member 100.

[0034] <Another configuration example 2 of the optical system of the space floating image display device> Another example of the configuration of the optical system of the space floating image display device will be described with reference to Fig. 2C. In Fig. 2C, the components with the same reference numerals as Fig. 2B have the same functions and configurations as Fig. 2B. For the sake of simplicity, the description of such components will not be repeated.

[0035] The only difference between the optical system in Fig. 2B and the optical system in Fig. 2C is the arrangement angle of the polarization separation member 101B with respect to the image display surface of the display device 1 and the surface of the retroreflector 2. All other configurations are the same as those of the optical system in Fig. 2B, so repeated explanations will be omitted. The polarization design of the optical system in Fig. 2C is also the same as that of the optical system in Fig. 2B, so repeated explanations will be omitted.

[0036] In the optical system of FIG. 2C, the polarization separation member 101B is arranged at an angle α with respect to the image display surface of the display device 1 and the surface of the retroreflector 2. In FIG. 2C, the angle α is 45°. With this configuration, in the reflection of the polarization separation member 101B, the angle β between the traveling direction of the image light reflected by the polarization separation member 101B (direction of the chief ray of the image light) and the traveling direction of the image light incident from the retroreflector 2 (direction of the chief ray of the image light) is 90°. With this configuration, the image display surface of the display device 1 and the surface of the retroreflector 2 are perpendicular to the traveling direction of the image light reflected by the polarization separation member 101B, and the angular relationship of the surfaces constituting the optical system can be simplified. If the surface of the transparent member 100 is arranged so as to be perpendicular to the traveling direction of the image light reflected by the polarization separation member 101B, the angular relationship of the surfaces constituting the optical system can be further simplified. In the configuration of FIG. 2C, when a user views the image from the direction of the arrow A, the floating image 3 is perceived as a bright image. However, when another person views the image from the direction of the arrow B, the floating image 3 cannot be seen as an image at all. This characteristic is very suitable for use in a system that displays images that require high security or highly confidential images that should be concealed from people directly facing the user.

[0037] As described above, the optical system of Fig. 2C can form a suitable floating image in space similar to the optical system of Fig. 2A and Fig. 2B, even though the optical system of Fig. 2C has a different configuration from the optical system of Fig. 2A and Fig. 2B. In addition, the angles of the surfaces constituting the optical system can be made simpler.

[0038] An absorptive polarizing plate may be provided on the surface of the transparent member 100 facing the polarization separation member 101B. The absorptive polarizing plate may transmit the polarized wave of the image light from the polarization separation member 101B and absorb the polarized wave that is 90° out of phase with the polarized wave of the image light from the polarization separation member 101B. In this way, the image light for forming the space-floating image 3 is sufficiently transmitted, while the external light incident from the space-floating image 3 side of the transparent member 100 can be reduced by about 50%. This makes it possible to reduce stray light in the optical system of FIG. 2C due to the external light incident from the space-floating image 3 side of the transparent member 100.

[0039] <Other Configuration Example 3 of the Optical System of the Space Floating Image Display Device> Another example of the configuration of the optical system of the space floating image display device will be described with reference to FIG. 2D. The optical system of FIG. 2D is an optical system using a retroreflector 5 different from the retroreflector 2 used in FIG. 2A to FIG. 2C. Another example of the configuration 3 of the optical system will be described in more detail below with reference to FIG. 2D to FIG. 2I. In FIG. 2D, components with the same reference numerals as those in FIG. 2A to FIG. 2C have the same functions and configurations. Such components will not be described repeatedly in order to simplify the explanation.

[0040] 2D is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention. A display device 10 that emits image light is provided in an oblique direction of a transparent member 100 such as glass. The display device 10 includes a liquid crystal display panel 11 and a light source device 13 that generates light.

[0041] A chief ray 9020 representing a light beam emitted from the display device 10 travels toward the retroreflector 5 and is incident on the retroreflector 5 at an incident angle α. The incident angle α may be, for example, 45°. However, the incident angle α is not limited to 45°, and may be, for example, 45°±15°.

[0042] The retroreflector 5 is an optical member having an optical property of retroreflecting light rays in at least some directions. In addition, since the reflected light rays have an optical property of forming an image, the retroreflector 5 may be expressed as an imaging optical member or an imaging optical plate.

[0043] 2E, 2F, etc., the principal ray 9020 travels in the z direction and is retroreflected in the x and y directions by the retroreflector 5. As a result, the reflected ray 9021 travels in a direction away from the retroreflector 5 along an optical path mirror-symmetrical to the principal ray 9020 with respect to the retroreflector 5, passes through the transparent member 100, and forms the floating image 3 as a real image on the imaging plane.

[0044] The light beam forming the floating image 3 is a collection of light rays that converge from the retroreflector 5 to the optical image of the floating image 3, and these light rays continue to travel straight even after passing through the optical image of the floating image 3. Therefore, the floating image 3 is an image with high directivity, unlike a diffuse image formed on a screen by a general projector or the like. Therefore, in the configuration of FIG. 2, when a user views the floating image 3 from the direction of the arrow A, the floating image 3 is viewed as a bright image. However, when another person views the floating image 3 from the direction of the arrow B, the floating image 3 cannot be viewed as an image at all. This characteristic is suitable for use in a system that displays images that require high security or highly confidential images that should be concealed from people facing the user.

[0045] An example of the configuration of the retroreflector 5 will be described with reference to Figs. 2E and 2F. The retroreflector 5 has a configuration in which a plurality of corner reflectors 9040 are arranged in an array on the surface of a transparent member 50. This may be called a corner reflector array or a multi-surface reflector array. The specific configuration of the corner reflector 9040 will be described in detail with reference to Figs. 2G, 2H, and 2I. The light rays 9111, 9112, 9113, and 9114 emitted from the light source 9110 are reflected twice by the two mirror surfaces 9041 and 9042 of the corner reflector 9040, and become reflected light rays 9121, 9122, 9123, and 9124. This double reflection is a retroreflection that turns back in the same direction as the incident direction (travels in a direction rotated by 180°) in the x and y directions, and a regular reflection in which the incident angle and the reflection angle match due to total reflection in the z direction.

[0046] That is, the light rays 9111 to 9114 generate reflected light rays 9121 to 9124 on a straight line symmetrical in the z direction with respect to the corner reflector 9040, and form an aerial real image 9120. The light rays 9111 to 9114 emitted from the light source 9110 are four light rays representative of the diffused light from the light source 9110, and although the light rays incident on the retroreflector 5 are not limited to these depending on the diffusion characteristics of the light source 9110, all the incident light rays cause similar reflection and form an aerial real image 9120. To make the drawing easier to see, the position of the light source 9110 and the position of the aerial real image 9120 in the x direction are shown shifted, but in reality the position of the light source 9110 and the position of the aerial real image 9120 in the x direction are the same position, and are overlapped when viewed from the z direction.

[0047] Next, the configuration and effect of the corner reflector 9040 constituting the retroreflector 5 will be described with reference to Figures 2G, 2H, and 2I. The corner reflector 9040 is a rectangular parallelepiped with only two specific faces being mirror faces 9041 and 9042, and the other four faces being made of transparent materials. The retroreflector 5 has a configuration in which the corner reflectors 9040 are arrayed so that the corresponding mirror faces face the same direction.

[0048] When viewed from the top (+z direction), a light ray 9111 emitted from a light source 9110 enters a mirror surface 9041 (or a mirror surface 9042) at a specific angle of incidence, is totally reflected at a reflection point 9130, and then is totally reflected again at a reflection point 9132 on the mirror surface 9042 (or a mirror surface 9041).

[0049] If the angle of incidence of the light ray 9111 with respect to the mirror surface 9041 (or the mirror surface 9042) is θ, the angle of incidence of the first reflected light ray 9131 reflected by the mirror surface 9041 (or the mirror surface 9042) with respect to the mirror surface 9042 (or the mirror surface 9041) can be expressed as 90°-θ. Therefore, with respect to the light ray 9111, the second reflected light ray 9121 is rotated by 2θ by the first reflection and by 2×(90°-θ) by the second reflection, resulting in a total inversion optical path of 180°. On the other hand, when viewed from the side (the direction halfway between -x and -y), total reflection in the z direction occurs only once. Therefore, with respect to the light ray 9111, if the angle of incidence with respect to the mirror surface 9041 or the mirror surface 9042 is φ, the reflected light ray 9121 is rotated by 2×φ by one reflection.

[0050] As a result, the light beam incident on the corner reflector 9040 undergoes retroreflection with an inverted optical path in the x and y directions, and undergoes regular reflection due to total reflection in the z direction. Considering the retroreflector 5, similar reflections occur in each optical path, so that an image is formed at a point symmetrical with respect to the z axis direction due to an inverted optical path that has convergence in the x and y directions.

[0051] 2A to 2C, the retroreflector 2 has retroreflection characteristics in three axial directions. As a result, when an incident light beam with diffusivity is incident on the retroreflector 2, the reflected light beam with convergence travels toward the side of the retroreflector 2 where the light source of the incident light is present. The reflected light beam with convergence forms an image in the air to form a space floating image 3. The traveling direction of the main ray of the reflected light beam with convergence reflected from the retroreflector 2 is the opposite direction to the traveling direction of the main ray of the incident light beam with diffusivity that is incident on the retroreflector 2.

[0052] 2D, the retroreflector 5 has biaxial retroreflection properties and is specular in the other axial direction. As a result, when a diffusive incident light beam is incident on the retroreflector 5, the convergent reflected light beam reflected by the corner reflector array travels toward the side opposite to the side where the light source of the incident light is present with respect to the retroreflector 5. The convergent reflected light beam forms an image in the air to form the space floating image 3.

[0053] The traveling direction of the chief ray of the convergent reflected light beam reflected by the corner reflector array of the retroreflector 5 is not the opposite direction to the traveling direction of the chief ray of the diffusive incident light beam incident on the retroreflector 5. The normal direction component of the plate-shaped surface of the retroreflector 5 in the traveling direction of the chief ray of the diffusive incident light beam incident on the retroreflector 5 and the normal direction component of the plate-shaped surface of the retroreflector 5 in the traveling direction of the chief ray after being reflected by the retroreflector 5 to become a convergent reflected light beam proceed in a straight line without changing before and after reflection by the corner reflector array.

[0054] That is, the diffusive incident light beam is converted into a convergent reflected light beam by reflection on the retroreflector 5, but in the normal direction of the plate-shaped surface of the retroreflector 5, the light beam travels as if passing through the retroreflector 5. Here, the diffusive incident light beam incident on the retroreflector 5 and the convergent reflected light beam emerging from the retroreflector 5 are geometrically symmetrical with respect to the plate-shaped surface of the retroreflector 5.

[0055] The resolution of the space-floating image formed by the light beam from the image output unit 10 depends heavily on the diameter D and pitch P (not shown) of the retroreflector 5 shown in Figures 2E and 2F, in addition to the resolution of the liquid crystal display panel 11. For example, when a 7-inch WUXGA (1920 x 1200 pixels) liquid crystal display panel is used, even if one pixel (one triplet) is about 80 μm, if the diameter D of the retroreflector is 240 μm and the pitch P is 300 μm, one pixel of the space-floating image will be equivalent to 300 μm. For this reason, the effective resolution of the space-floating image is reduced to about 1 / 3.

[0056] Therefore, in order to make the resolution of the spatial floating image equal to that of the display device 10, it is desirable to make the diameter D and pitch P of the retroreflective portion close to one pixel of the liquid crystal display panel. On the other hand, in order to suppress the occurrence of moire due to the retroreflective plate and the pixels of the liquid crystal display panel, it is preferable to design the pitch ratio of each to be a different integer multiple of one pixel. In addition, it is preferable to arrange the shape so that none of the sides of the retroreflective portion overlaps with any of the sides of one pixel of the liquid crystal display panel.

[0057] The shape of the retroreflector (imaging optical plate) according to this embodiment is not limited to the above example. It may have various shapes that realize retroreflection. Specifically, it may be various cubic corner bodies, corner reflector arrays, slit mirror arrays, two-sided corner reflector arrays, polyhedral reflector arrays, or shapes in which a combination of their reflective surfaces is periodically arranged. Alternatively, a capsule lens type retroreflection element in which glass beads are periodically arranged may be provided on the surface of the retroreflection plate of this embodiment. Since the detailed configuration of these retroreflection elements can be achieved by using existing technology, detailed description will be omitted. Specifically, it is possible to use the technology disclosed in JP2017-33005A, JP2019-133110A, JP2017-67933A, WO2009 / 131128A, etc.

[0058] 2D, the image light emitted from the display device 10 may be in any polarization state. It does not matter whether it is S-polarized or P-polarized.

[0059] As described above, the optical system of FIG. 2D is an optical system that uses a different retroreflector from the optical systems of FIGS. 2A to 2C, but it can form a more suitable floating-in-space image, similar to the optical system of FIGS. 2A to 2C.

[0060] According to the optical systems of FIGS. 2A, 2B, 2C, and 2D described above, it is possible to provide brighter and higher quality floating images in space.

[0061] <<Block diagram of the internal configuration of the space floating image display device>>

[0062] Next, a description will be given of a block diagram of the internal configuration of the space-floating image display device 1000. Fig. 3 is a block diagram showing an example of the internal configuration of the space-floating image display device 1000.

[0063] The space floating image display device 1000 includes a retroreflection unit 1101, an image display unit 1102, a light guide 1104, a light source 1105, a power source 1106, an external power source input interface 1111, an operation input unit 1107, a non-volatile memory 1108, a memory 1109, a control unit 1110, an image signal input unit 1131, an audio signal input unit 1133, a communication unit 1132, an air operation detection sensor 1351, an air operation detection unit 1350, an audio output unit 1140, a microphone 1139, an image control unit 1160, a storage unit 1170, an imaging unit 1180, etc. In addition, the space floating image display device 1000 may include a removable media interface 1134, an attitude sensor 1113, a transmissive self-luminous image display device 1650, a second display device 1680, or a secondary battery 1112.

[0064] Each component of the space floating image display device 1000 is disposed in a housing 1190. Note that the imaging unit 1180 and the mid-air operation detection sensor 1351 shown in FIG.

[0065] The retroreflector 1101 in Fig. 3 corresponds to the retroreflector 2 in Fig. 2A, Fig. 2B, and Fig. 2C. The retroreflector 1101 retroreflects light modulated by the image display unit 1102. Of the light reflected from the retroreflector 1101, the light outputted to the outside of the space-floating image display device 1000 forms the space-floating image 3.

[0066] Image display unit 1102 in Fig. 3 corresponds to liquid crystal display panel 11 in Fig. 2A, Fig. 2B, and Fig. 2C. Light source 1105 in Fig. 3 corresponds to light source device 13 in Fig. 2A, Fig. 2B, and Fig. 2C. Image display unit 1102, light guide 1104, and light source 1105 in Fig. 3 correspond to display device 1 in Fig. 2A, Fig. 2B, and Fig. 2C.

[0067] The video display unit 1102 is a display unit that generates a video by modulating transmitted light based on a video signal input under the control of a video control unit 1160 described later. The video display unit 1102 corresponds to the liquid crystal display panel 11 in Figs. 2A, 2B, and 2C. For example, a transmissive liquid crystal panel is used as the video display unit 1102. Alternatively, for example, a reflective liquid crystal panel that modulates reflected light or a DMD (Digital Micromirror Device: registered trademark) panel may be used as the video display unit 1102.

[0068] The light source 1105 generates light for the image display unit 1102 and is a solid-state light source such as an LED light source or a laser light source. The power source 1106 converts an AC current input from the outside via the external power input interface 1111 into a DC current and supplies power to the light source 1105. The power source 1106 also supplies the necessary DC current to each unit in the space-floating image display device 1000. The secondary battery 1112 stores the power supplied from the power source 1106. The secondary battery 1112 also supplies power to the light source 1105 and other components that require power when power is not supplied from the outside via the external power input interface 1111. That is, when the space-floating image display device 1000 includes the secondary battery 1112, the user can use the space-floating image display device 1000 even when power is not supplied from the outside.

[0069] The light guide 1104 guides the light generated by the light source 1105 and irradiates the image display unit 1102. The combination of the light guide 1104 and the light source 1105 can be called a backlight of the image display unit 1102. The light guide 1104 may be mainly made of glass. The light guide 1104 may be mainly made of plastic. The light guide 1104 may be made of a mirror. There are various types of combination of the light guide 1104 and the light source 1105. Specific configuration examples of the combination of the light guide 1104 and the light source 1105 will be described in detail later.

[0070] The aerial operation detection sensor 1351 is a sensor that detects an operation of the floating-in-space image 3 by a finger of the user 230. The aerial operation detection sensor 1351 senses, for example, a range that overlaps with the entire display range of the floating-in-space image 3. Note that the aerial operation detection sensor 1351 may only sense a range that overlaps with at least a portion of the display range of the floating-in-space image 3.

[0071] Specific examples of the aerial operation detection sensor 1351 include a distance sensor using invisible light such as infrared light, an invisible light laser, ultrasonic waves, etc. The aerial operation detection sensor 1351 may be configured to detect coordinates on a two-dimensional plane by combining a plurality of sensors. The aerial operation detection sensor 1351 may be configured with a ToF (Time of Flight) LiDAR (Light Detection and Ranging) or an image sensor.

[0072] The mid-air operation detection sensor 1351 only needs to be capable of sensing to detect a touch operation or the like performed by a user with a finger on an object displayed as the floating-in-space image 3. Such sensing can be performed using existing technology.

[0073] The aerial operation detection unit 1350 acquires a sensing signal from the aerial operation detection sensor 1351, and performs operations such as determining whether or not the finger of the user 230 has touched an object in the floating in space image 3 and calculating the position (contact position) where the finger of the user 230 has touched the object based on the sensing signal. The aerial operation detection unit 1350 is configured with a circuit such as an FPGA (Field Programmable Gate Array). Some of the functions of the aerial operation detection unit 1350 may be realized by software using a spatial operation detection program executed by the control unit 1110, for example.

[0074] The aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be configured to be built into the space-floating image display device 1000, or may be provided externally separately from the space-floating image display device 1000. When provided separately from the space-floating image display device 1000, the aerial operation detection sensor 1351 and the aerial operation detection unit 1350 are configured to transmit information and signals to the space-floating image display device 1000 via a wired or wireless communication connection path or image signal transmission path.

[0075] Also, the aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be provided separately. This makes it possible to build a system in which the air-floating image display device 1000 without the aerial operation detection function is used as the main body, and only the aerial operation detection function can be added as an option. Also, a configuration in which only the aerial operation detection sensor 1351 is provided separately, and the aerial operation detection unit 1350 is built into the air-floating image display device 1000 may be used. In cases in which it is desired to more freely arrange the aerial operation detection sensor 1351 relative to the installation position of the air-floating image display device 1000, a configuration in which only the aerial operation detection sensor 1351 is provided separately is advantageous.

[0076] The imaging unit 1180 is a camera having an image sensor, and captures the space near the floating-in-space image 3 and / or the face, arms, fingers, etc. of the user 230. A plurality of imaging units 1180 may be provided. By using a plurality of imaging units 1180 or an imaging unit with a depth sensor, the mid-air operation detection unit 1350 can be assisted in the detection process of the touch operation of the floating-in-space image 3 by the user 230. The imaging unit 1180 may be provided separately from the floating-in-space image display device 1000. When the imaging unit 1180 is provided separately from the floating-in-space image display device 1000, it is sufficient to configure it so that an imaging signal can be transmitted to the floating-in-space image display device 1000 via a wired or wireless communication connection path or the like.

[0077] For example, if the aerial operation detection sensor 1351 is configured as an object intrusion sensor that detects whether or not an object has intruded into a plane (intrusion detection plane) including the display surface of the floating image 3, the aerial operation detection sensor 1351 may not be able to detect information such as how far an object (e.g., a user's finger) that has not intruded into the intrusion detection plane is from the intrusion detection plane, or how close the object is to the intrusion detection plane.

[0078] In such a case, the distance between the object and the intrusion detection plane can be calculated by using information such as object depth calculation information based on the captured images of the multiple imaging units 1180 and object depth information by the depth sensor. These pieces of information and various pieces of information such as the distance between the object and the intrusion detection plane are used for various display controls for the floating in space image 3.

[0079] Furthermore, without using the aerial operation detection sensor 1351, the aerial operation detection section 1350 may detect a touch operation on the floating-in-space image 3 by the user 230 based on the captured image by the imaging section 1180.

[0080] Also, the imaging unit 1180 may capture an image of the face of the user 230 operating the floating image 3, and the control unit 1110 may perform an identification process for the user 230. Furthermore, in order to determine whether or not a third person is standing around or behind the user 230 operating the floating image 3 and peeking at the operation of the user 230 on the floating image 3, the imaging unit 1180 may capture an image of a range including the user 230 operating the floating image 3 and the surrounding area of ​​the user 230.

[0081] The operation input unit 1107 is, for example, an operation button, a signal receiving unit such as a remote controller, or an infrared light receiving unit, and inputs a signal for an operation different from the air operation (touch operation) by the user 230. In addition to the above-mentioned user 230 who touches the space floating image 3, the operation input unit 1107 may be used, for example, by an administrator to operate the space floating image display device 1000.

[0082] The video signal input unit 1131 is connected to an external video output device and inputs video data. Various digital video input interfaces are possible for the video signal input unit 1131. For example, the video signal input unit 1131 may be configured with a video input interface conforming to the HDMI (registered trademark) (High-Definition Multimedia Interface) standard, a video input interface conforming to the DVI (Digital Visual Interface) standard, or a video input interface conforming to the DisplayPort standard.

[0083] Alternatively, an analog video input interface such as analog RGB or composite video may be provided. The audio signal input unit 1133 inputs audio data by connecting an external audio output device. The audio signal input unit 1133 may be configured with an audio input interface conforming to the HDMI standard, an optical digital terminal interface, a coaxial digital terminal interface, or the like. In the case of an interface conforming to the HDMI standard, the video signal input unit 1131 and the audio signal input unit 1133 may be configured as an interface in which a terminal and a cable are integrated. The audio output unit 1140 is capable of outputting audio based on the audio data input to the audio signal input unit 1133. The audio output unit 1140 may be configured with a speaker.

[0084] The audio output unit 1140 may output built-in operation sounds or error warning sounds. Alternatively, the audio output unit 1140 may be configured to output a digital signal to an external device, like the Audio Return Channel function defined in the HDMI standard. The microphone 1139 is a microphone that collects sounds around the space floating image display device 1000, converts them into signals, and generates audio signals. The microphone may record a person's voice, such as a user's voice, and the control unit 1110, which will be described later, may perform voice recognition processing on the generated audio signal to obtain text information from the audio signal.

[0085] The non-volatile memory 1108 stores various data used by the space-floating image display device 1000. The data stored in the non-volatile memory 1108 includes, for example, data for various operations to be displayed on the space-floating image 3, display icons, data of objects to be operated by user operations, layout information, etc. The memory 1109 stores image data to be displayed as the space-floating image 3, data for controlling the device, etc.

[0086] The control unit 1110 controls the operation of each unit connected to it. In addition, the control unit 1110 may perform calculation processing based on information obtained from each unit in the space floating image display device 1000 in cooperation with a program stored in the memory 1109.

[0087] The communication unit 1132 communicates with external devices, external servers, and the like via a wired or wireless communication interface. When the communication unit 1132 has a wired communication interface, the wired communication interface may be, for example, an Ethernet LAN interface. When the communication unit 1132 has a wireless communication interface, the communication unit 1132 may be, for example, a Wi-Fi communication interface, a Bluetooth communication interface, or a mobile communication interface such as 4G or 5G. Various data such as video data, image data, and audio data are transmitted and received by communication via the communication unit 1132.

[0088] The removable media interface 1134 is an interface for connecting a removable recording medium (removable media). The removable recording medium (removable media) may be composed of a semiconductor element memory such as a solid state drive (SSD), a magnetic recording medium recording device such as a hard disk drive (HDD), or an optical recording medium such as an optical disk. The removable media interface 1134 is capable of reading out various information such as various data such as video data, image data, and audio data recorded on the removable recording medium. The video data, image data, and the like recorded on the removable recording medium are output as a space floating image 3 via the video display unit 1102 and the retroreflection unit 1101.

[0089] The storage unit 1170 is a storage device that records various information such as various data such as video data, image data, and audio data. The storage unit 1170 may be configured with a magnetic recording medium recording device such as a hard disk drive (HDD) or a semiconductor element memory such as a solid state drive (SSD). For example, various information such as various data such as video data, image data, and audio data may be recorded in advance in the storage unit 1170 at the time of product shipment. In addition, the storage unit 1170 may record various information such as various data such as video data, image data, and audio data acquired from an external device, an external server, etc. via the communication unit 1132.

[0090] The video data, image data, etc. recorded in the storage unit 1170 are output as the space floating image 3 via the video display unit 1102 and the retroreflection unit 1101. The video data, image data, etc. of the display icons and objects for the user to operate, which are displayed as the space floating image 3, are also recorded in the storage unit 1170.

[0091] Layout information of display icons and objects displayed as the space floating image 3, various metadata information related to the objects, and the like are also recorded in the storage unit 1170. The audio data recorded in the storage unit 1170 is output as audio from the audio output unit 1140, for example.

[0092] The video control unit 1160 performs various controls related to the video signal input to the video display unit 1102. The video control unit 1160 may be called a video processing circuit, and may be configured with hardware such as an ASIC, an FPGA, or a video processor. The video control unit 1160 may also be called a video processing unit or an image processing unit. The video control unit 1160 performs control of video switching, such as determining which video signal to input to the video display unit 1102, between the video signal to be stored in the memory 1109 and the video signal (video data) input to the video signal input unit 1131, for example.

[0093] In addition, the image control unit 1160 may generate a superimposed image signal by superimposing the image signal to be stored in memory 1109 and the image signal input from the image signal input unit 1131, and input the superimposed image signal to the image display unit 1102, thereby performing control to form a composite image as a floating-in-space image 3.

[0094] Furthermore, the video control unit 1160 may control image processing of the video signal input from the video signal input unit 1131, the video signal to be stored in the memory 1109, etc. Examples of image processing include scaling processing for enlarging, reducing, transforming, etc. an image, brightness adjustment processing for changing the luminance, contrast adjustment processing for changing the contrast curve of the image, and Retinex processing for decomposing an image into light components and changing the weighting of each component.

[0095] Furthermore, the video control unit 1160 may perform special effect video processing or the like for assisting the aerial operation (touch operation) of the user 230 on the video signal input to the video display unit 1102. The special effect video processing is performed, for example, based on the detection result of the touch operation of the user 230 by the aerial operation detection unit 1350 or the captured image of the user 230 by the imaging unit 1180.

[0096] The attitude sensor 1113 is a sensor composed of a gravity sensor, an acceleration sensor, or a combination of these, and can detect the attitude in which the space-floating image display device 1000 is installed. Based on the attitude detection result of the attitude sensor 1113, the control unit 1110 may control the operation of each connected unit. For example, when an undesirable attitude as a user's usage state is detected, the control unit 1110 may perform control to stop displaying the image displayed by the image display unit 1102 and display an error message to the user. Alternatively, when the attitude sensor 1113 detects that the installation attitude of the space-floating image display device 1000 has changed, the control unit 1110 may perform control to rotate the display direction of the image displayed by the image display unit 1102.

[0097] As explained above, various functions are installed in the space-floating image display device 1000. However, the space-floating image display device 1000 does not need to have all of these functions, and can have any configuration as long as it has the function of forming the space-floating image 3.

[0098] <Example of the configuration of a space floating image display device> Next, a configuration example of the space-floating image display device will be described. The layout of the components of the space-floating image display device according to this embodiment can be various depending on the usage form. Each layout of Fig. 4A to Fig. 4M will be described below. In addition, in each example of Fig. 4A to Fig. 4M, the thick line surrounding the space-floating image display device 1000 shows an example of the housing structure of the space-floating image display device 1000.

[0099] FIG. 4A is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4A is equipped with an optical system corresponding to the optical system of FIG. 2A. The space-floating image display device 1000 shown in FIG. 4A is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4A, the space-floating image display device 1000 has a transparent member 100 installed on the upper surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. When the mid-air operation detection sensor 1351 is provided as shown in the figure, it is possible to detect the operation of the space-floating image 3 by the finger of the user 230. Note that the x direction is the left-right direction as seen from the user, the y direction is the front-back direction (depth direction) as seen from the user, and the z direction is the up-down direction (vertical direction). Hereinafter, the definitions of the x-direction, y-direction, and z-direction are the same in each drawing of FIG. 4, so repeated explanation will be omitted.

[0100] FIG. 4B is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4B is equipped with an optical system corresponding to the optical system of FIG. 2A. The space-floating image display device 1000 shown in FIG. 4B is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (the direction of the user 230). That is, in FIG. 4B, the space-floating image display device is installed with the transparent member 100 on the front side of the device (the direction of the user 230). The space-floating image 3 is formed on the user 230 side with respect to the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels in an oblique upward direction. When the midair operation detection sensor 1351 is provided as shown in the figure, it is possible to detect the operation of the space-floating image 3 by the finger of the user 230. 4B, the mid-air operation detection sensor 1351 can use the reflection of sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, the nail has a higher reflectivity than the pad of the finger, so that the accuracy of touch detection can be improved by configuring in this way.

[0101] FIG. 4C is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4C is equipped with an optical system corresponding to the optical system of FIG. 2B. The space-floating image display device 1000 shown in FIG. 4C is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4C, the space-floating image display device 1000 has a transparent member 100 installed on the upper surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. When the mid-air operation detection sensor 1351 is provided as shown in the figure, it is possible to detect the operation of the space-floating image 3 by the finger of the user 230.

[0102] FIG. 4D is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4D is equipped with an optical system corresponding to the optical system of FIG. 2B. The space-floating image display device 1000 shown in FIG. 4D is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (the direction of the user 230). That is, in FIG. 4D, the space-floating image display device 1000 is installed with the transparent member 100 on the front side of the device (the direction of the user 230). The space-floating image 3 is formed on the user 230 side with respect to the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels in an oblique upward direction. When the midair operation detection sensor 1351 is provided as shown in the figure, it is possible to detect the operation of the space-floating image 3 by the finger of the user 230. 4D, the mid-air operation detection sensor 1351 can use the reflection of sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, the nail has a higher reflectivity than the pad of the finger, so that the accuracy of touch detection can be improved by configuring in this way.

[0103] FIG. 4E is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4E is equipped with an optical system corresponding to the optical system of FIG. 2C. The space-floating image display device 1000 shown in FIG. 4E is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4E, the space-floating image display device 1000 has a transparent member 100 installed on the upper surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels directly upward. When the mid-air operation detection sensor 1351 is provided as shown in the figure, it is possible to detect the operation of the space-floating image 3 by the finger of the user 230.

[0104] FIG. 4F is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4F is equipped with an optical system corresponding to the optical system of FIG. 2C. The space-floating image display device 1000 shown in FIG. 4F is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (the direction of the user 230). That is, in FIG. 4F, the space-floating image display device 1000 is installed with the transparent member 100 on the front side of the device (the direction of the user 230). The space-floating image 3 is formed on the user 230 side with respect to the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels toward the user. When the midair operation detection sensor 1351 is provided as shown in the figure, it is possible to detect the operation of the space-floating image 3 by the finger of the user 230.

[0105] FIG. 4G is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4G is equipped with an optical system corresponding to the optical system of FIG. 2C. In the optical system of the space-floating image display device shown in FIG. 4A to FIG. 4F, the central optical path of the image light emitted from the display device 1 was on the yz plane. That is, in the optical system of the space-floating image display device shown in FIG. 4A to FIG. 4F, the image light traveled in the front-back direction and the up-down direction as seen from the user. In contrast, in the optical system of the space-floating image display device shown in FIG. 4G, the central optical path of the image light emitted from the display device 1 is on the xy plane. That is, in the optical system of the space-floating image display device shown in FIG. 4G, the image light travels in the left-right direction and the front-back direction as seen from the user. In the space-floating image display device 1000 shown in FIG. 4G, the surface on the side on which the space-floating image 3 is formed is installed so as to face the front of the device (the direction of the user 230). That is, in Fig. 4G, the space-floating image display device 1000 has the transparent member 100 installed on the front side of the device (toward the user 230). The space-floating image 3 is formed on the user side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels toward the user. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230.

[0106] FIG. 4H is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4H is different from the space-floating image display device of FIG. 4G in that it has a window having a transparent plate 100B such as glass or plastic on the back side of the device (opposite the position where the user 230 views the space-floating image 3, i.e., opposite the traveling direction of the image light of the space-floating image 3 toward the user 230). The other configurations are the same as those of the space-floating image display device of FIG. 4G, so repeated explanations will be omitted. The space-floating image display device 1000 of FIG. 4H has a window having a transparent plate 100B at a position opposite the traveling direction of the image light of the space-floating image 3 with respect to the space-floating image 3. Therefore, when the user 230 views the space-floating image 3, the user 230 can recognize the scenery behind the space-floating image display device 1000 as the background of the space-floating image 3. Therefore, the user 230 can recognize the space floating image 3 as floating in the air in front of the scenery behind the space floating image display device 1000. This can further emphasize the feeling of floating in the air of the space floating image 3.

[0107] Depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, a part of the image light output from the display device 1 may be reflected by the polarization separation member 101B and directed toward the transparent plate 100B. Depending on the coating performance of the surface of the transparent plate 100B, this light may be reflected again by the surface of the transparent plate 100B and may be visually recognized by the user as stray light. Therefore, in order to prevent this stray light, the transparent plate 100B may not be provided on the window on the back of the space floating image display device 1000.

[0108] Fig. 4I is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 in Fig. 4I is different from the space-floating image display device in Fig. 4H in that a light-shielding door 1410 is provided in the window of the transparent plate 100B arranged on the back of the device (opposite the position where the user 230 views the space-floating image 3). The other configurations are the same as those of the space-floating image display device in Fig. 4H, so repeated explanations will be omitted.

[0109] The opening and closing door 1410 of the space-floating image display device 1000 in FIG. 4I has, for example, a light-shielding plate, and is provided with a mechanism for moving (sliding) the light-shielding plate, a mechanism for rotating the light-shielding plate, or a mechanism for attaching and detaching the light-shielding plate, so that the window (rear window) of the transparent plate 100B located at the back side of the space-floating image display device 1000 can be switched between an open state and a light-shielding state. The movement (sliding) and rotation of the light-shielding plate by the opening and closing door 1410 may be electrically driven by a motor (not shown). The motor may be controlled by the control unit 1110 in FIG. 3. Note that, in the example of FIG. 4I, an example in which the opening and closing door 1410 has two light-shielding plates is disclosed. In contrast, the number of light-shielding plates of the opening and closing door 1410 may be one.

[0110] For example, when the scenery seen through the window of the transparent plate 100B of the space-floating image display device 1000 is outdoors, the brightness of sunlight varies depending on the weather. When the sunlight outdoors is strong, the background of the space-floating image 3 may become too bright, and the user 230 may experience a decrease in visibility of the space-floating image 3. In such a case, if the rear window is put into a light-shielding state by moving (sliding), rotating, or attaching the light-shielding plate of the opening and closing door 1410, the background of the space-floating image 3 becomes dark, and the visibility of the space-floating image 3 can be relatively improved. Such a shielding operation by the light-shielding plate of the opening and closing door 1410 may be directly performed by the force of the hand of the user 230. In response to an operation input via the operation input unit 1107 of FIG. 3, the control unit 1110 may control a motor (not shown) to perform a shielding operation by the light-shielding plate of the opening and closing door 1410.

[0111] In addition, an illuminance sensor may be provided on the rear side (opposite side of the user 230) of the space-floating image display device 1000, such as near the rear window, to measure the brightness of the space beyond the rear window. In this case, the control unit 1110 in FIG. 3 may control a motor (not shown) to perform the opening and closing operation of the light shielding plate of the opening and closing door 1410 according to the detection result of the illuminance sensor. By controlling the opening and closing operation of the light shielding plate of the opening and closing door 1410 in this way, it is possible to more suitably maintain the visibility of the space-floating image 3 even if the user 230 does not manually open and close the light shielding plate of the opening and closing door 1410.

[0112] Also, the light shielding plate by the opening and closing door 1410 may be manually detachable. Depending on the use purpose and installation environment of the space floating image display device 1000, the user can select whether the rear window is opened or shielded. If the rear window is to be used in a light shielded state for a long period of time, the removable light shielding plate may be fixed in the light shielded state. If the rear window is to be used in an open state for a long period of time, the removable light shielding plate may be removed. The light shielding plate may be attached and detached using a screw, a hook structure, or a fitting structure.

[0113] In the example of the space-floating image display device 1000 in FIG. 4I, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, a part of the image light output from the display device 1 may be reflected by the polarization separation member 101B and directed toward the transparent plate 100B. Depending on the coating performance of the surface of the transparent plate 100B, this light may be reflected again by the surface of the transparent plate 100B and may be visually recognized by the user as stray light. Therefore, in order to prevent the stray light, the window on the back of the space-floating image display device 1000 may be configured without providing the transparent plate 100B. The above-mentioned opening and closing door 1410 may be provided on the window that does not have the transparent plate 100B. In order to prevent the stray light, it is desirable that the surface of the light shielding plate of the above-mentioned opening and closing door 1410 on the inside of the housing has a coating or material with low light reflectance.

[0114] Fig. 4J is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4J is different from the space-floating image display device of Fig. 4H in that an electronically controlled transmittance variable device 1620 is arranged on the rear side window instead of the transparent plate 100B made of glass or plastic. The other configurations are the same as those of the space-floating image display device of Fig. 4H, so repeated explanations will be omitted. An example of the electronically controlled transmittance variable device 1620 is a liquid crystal shutter.

[0115] That is, the liquid crystal shutter can control the transmitted light by controlling the voltage of the liquid crystal element sandwiched between two polarizing plates. Therefore, if the liquid crystal shutter is controlled to increase the transmittance, the scenery through the rear window can be seen through the background of the floating image 3. Also, if the liquid crystal shutter is controlled to increase the transmittance, the scenery through the rear window can be hidden as the background of the floating image 3. In addition, since the liquid crystal shutter can control the intermediate length, it can also be set to a state of transmittance of 50% or the like. For example, the control unit 1110 can control the transmittance of the electronically controlled transmittance variable device 1620 in response to an operation input via the operation input unit 1107 in Fig. 3. With this configuration, in cases where a view through the rear window is desired as the background of the floating image in space 3, but the view through the rear window as the background is too bright and reduces the visibility of the floating image in space 3, it is possible to adjust the transmittance of the electronically controlled transmittance variable device 1620 to adjust the visibility of the floating image in space 3.

[0116] In addition, an illuminance sensor may be provided on the back side (opposite side of the user 230) of the space floating image display device 1000, such as near the back window, to measure the brightness of the space beyond the back window. In this case, the control unit 1110 in Fig. 3 may control the transmittance of the electronically controlled transmittance variable device 1620 according to the detection result of the illuminance sensor. In this way, even if the user 230 does not perform an operation input via the operation input unit 1107 in Fig. 3, the transmittance of the electronically controlled transmittance variable device 1620 can be adjusted according to the brightness of the space beyond the back window, so that the visibility of the space floating image 3 can be more suitably maintained.

[0117] In the above example, a liquid crystal shutter has been described as an example of the electronically controlled transmittance variable device 1620. However, electronic paper may be used as another example of the electronically controlled transmittance variable device 1620. The same effect as described above can be obtained when electronic paper is used. Furthermore, electronic paper consumes very little power to maintain a halftone state. Therefore, a space floating image display device with low power consumption can be realized compared to the case where a liquid crystal shutter is used.

[0118] Fig. 4K is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4K is different from the space-floating image display device of Fig. 4G in that it has a transmissive self-luminous image display device 1650 instead of the transparent member 100. The other configurations are the same as those of the space-floating image display device of Fig. 4G, so repeated explanations will be omitted.

[0119] In the space-floating image display device 1000 of FIG. 4K, after the image light flux passes through the display surface of the transmissive self-luminous image display device 1650, the space-floating image 3 is formed outside the space-floating image display device 1000. That is, when an image is displayed on the transmissive self-luminous image display device 1650, which is a two-dimensional flat display, the space-floating image 3 can be displayed as a pop-out image further in front of the image of the transmissive self-luminous image display device 1650. At this time, the user 230 can simultaneously view two images at different depth positions. The transmissive self-luminous image display device 1650 may be configured using existing technology such as a transmissive organic EL panel disclosed in, for example, JP 2014-216761 A. Note that the transmissive self-luminous image display device 1650 is not shown in FIG. 3, but may be configured to be connected to other processing units such as the control unit 1110 as one component of the space-floating image display device 1000 of FIG. 3.

[0120] Here, if both the background and an object such as a character are displayed on the transmissive self-luminous image display device 1650, and then only the object such as the character moves to the foreground floating image 3, a more effective surprise video experience can be provided to the user 230.

[0121] Also, if the inside of the space-floating image display device 1000 is kept in a light-shielding state, the background of the transparent self-luminous image display device 1650 becomes sufficiently dark. Therefore, when no image is displayed on the display device 1, or the light source of the display device 1 is turned off, and an image is displayed only on the transparent self-luminous image display device 1650, the transparent self-luminous image display device 1650 appears to the user 230 as a normal two-dimensional flat display, not a transparent display (since the space-floating image 3 in the embodiment of the present invention is displayed as a real optical image in a space without a screen, if the light source of the display device 1 is turned off, the intended display position of the space-floating image 3 becomes an empty space). Therefore, when the transparent self-luminous image display device 1650 is used to display an image as if it were a general two-dimensional flat display, characters, objects, etc. can be suddenly displayed in the air as the space-floating image 3, providing the user 230 with a more effective video experience with a surprise production.

[0122] In addition, the darker the inside of the space-floating image display device 1000, the more the transmissive self-luminous image display device 1650 looks like a two-dimensional flat display. Therefore, an absorptive polarizing plate (not shown) that transmits the polarized wave of the image light reflected by the polarization separation member 101B and absorbs the polarized wave that is 90 degrees out of phase with the polarized wave may be provided on the inner surface of the transmissive self-luminous image display device 1650 (the incident surface of the image light reflected by the polarization separation member 101B to the transmissive self-luminous image display device 1650, i.e., the surface of the transmissive self-luminous image display device 1650 opposite to the space-floating image 3). In this way, the effect on the image light that forms the space-floating image 3 is not so large, but the light that enters the inside of the space-floating image display device 1000 from the outside through the transmissive self-luminous image display device 1650 can be significantly reduced, and the inside of the space-floating image display device 1000 can be made darker, which is preferable.

[0123] FIG. 4L is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4L is a modified example of the space-floating image display device of FIG. 4K. The orientation of the components in the space-floating image display device 1000 is different from that of the space-floating image display device of FIG. 4K, and is closer to the space-floating image display device of FIG. 4F. The functions and operations of each component are the same as those of the space-floating image display device of FIG. 4K, so repeated explanations will be omitted.

[0124] In the space-floating image display device of FIG. 4L as well, after a luminous flux of image light passes through the transmissive self-luminous image display device 1650, a space-floating image 3 is formed on the user 230 side of the transmissive self-luminous image display device 1650.

[0125] In both the example of the space-floating image display device in FIG. 4K and the example of the space-floating image display device in FIG. 4L, the space-floating image 3 is displayed superimposed on the image of the transmissive self-luminous image display device 1650 as seen by the user 230. Here, the position of the space-floating image 3 and the position of the image of the transmissive self-luminous image display device 1650 are configured to have a difference in the depth direction. Therefore, when the user moves his / her head (position of viewpoint), the user can recognize the depth of the two images due to parallax. Therefore, by displaying two images with different depth positions, a three-dimensional image experience can be more suitably provided to the user with the naked eye without the need for stereoscopic glasses or the like.

[0126] Fig. 4M is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4M is provided with a second display device 1680 on the rear side of the polarization separation member 101B of the space-floating image display device of Fig. 4G as seen from the user. The other configurations are the same as those of the space-floating image display device of Fig. 4G, so repeated explanations will be omitted.

[0127] In the configuration example shown in FIG. 4M, the second display device 1680 is provided behind the display position of the space-floating image 3, and the image display surface faces the space-floating image 3. With this configuration, the image of the second display device 1680 and the space-floating image 3, which are displayed at two different depth positions, can be viewed as overlapping images from the user 230's perspective. In other words, the second display device 1680 is arranged in a direction to display an image toward the user 230 who views the space-floating image 3. Although the second display device 1680 is not shown in FIG. 3, it may be configured to be connected to other processing units such as the control unit 1110 as one component of the space-floating image display device 1000 of FIG. 3.

[0128] The image light of the second display device 1680 of the space floating image display device 1000 of FIG. 4M is visually recognized by the user 230 after passing through the polarization separation member 101B. Therefore, in order for the image light of the second display device 1680 to pass through the polarization separation member 101B more preferably, it is desirable that the image light output from the second display device 1680 is polarized in a vibration direction that the polarization separation member 101B more preferably transmits. That is, it is desirable that the image light is polarized in the same vibration direction as the polarization of the image light output from the display device 1. For example, when the image light output from the display device 1 is S-polarized light, it is desirable that the image light output from the second display device 1680 is also S-polarized light. Furthermore, when the image light output from the display device 1 is P-polarized light, it is desirable that the image light output from the second display device 1680 is also P-polarized light.

[0129] The example of the space-floating image display device of FIG. 4M also has the same effect as the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L in that the second image is displayed behind the space-floating image 3. However, unlike the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L, in the example of the space-floating image display device of FIG. 4M, the luminous flux of the image light for forming the space-floating image 3 does not pass through the second display device 1680. Therefore, the second display device 1680 does not need to be a transmissive self-luminous image display device, and may be a liquid crystal display that is a two-dimensional flat display. The second display device 1680 may be an organic EL display. Therefore, in the example of the space-floating image display device of FIG. 4M, it is possible to realize the space-floating image display device 1000 at a lower cost than the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L.

[0130] Here, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, a part of the image light output from the display device 1 may be reflected by the polarization separation member 101B and travel toward the second display device 1680. This light (a part of the image light) may be reflected again by the surface of the second display device 1680 and may be visually recognized by the user as stray light.

[0131] Therefore, in order to prevent the stray light, an absorptive polarizing plate may be provided on the surface of the second display device 1680. In this case, the absorptive polarizing plate may be an absorptive polarizing plate that transmits the polarized wave of the image light output from the second display device 1680 and absorbs the polarized wave that is 90° out of phase with the polarized wave of the image light output from the second display device 1680. If the second display device 1680 is a liquid crystal display, an absorptive polarizing plate is also present on the image output side inside the liquid crystal display. However, if there is a cover glass (cover glass on the image display surface side) on the output surface of the absorptive polarizing plate on the image output side inside the liquid crystal display, it is not possible to prevent stray light caused by reflection of the cover glass due to light from outside the liquid crystal display. Therefore, it is necessary to separately provide the above-mentioned absorptive polarizing plate on the surface of the cover glass.

[0132] When an image is displayed on the second display device 1680, which is a two-dimensional flat display, the floating-in-space image 3 can be displayed as an image in front of the image on the second display device 1680. In this case, the user 230 can simultaneously view two images with different depth positions. By displaying a character on the floating-in-space image 3 and displaying a background on the second display device 1680, it is possible to provide an effect in which the user 230 is stereoscopically viewing the space in which the character exists.

[0133] Furthermore, if both the background and an object such as a character are displayed on the second display device 1680, and then only the object such as the character moves to the foreground floating image 3, it is possible to provide the user 230 with a more effective surprise visual experience.

[0134] Next, Fig. 4N is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 in Fig. 4N is a space-floating image display device that employs the optical system in Fig. 2D. As in the example of the space-floating image display device that employs the optical system in Figs. 2A to 2C, an image is formed in the air as a space-floating image 3 by image light that has passed through a transparent member 100. In addition, the operation of the space-floating image 3 by the user's finger 9004 can be detected using the sensing light of the mid-air operation detection sensor 1351 arranged on the back side of the transparent member 100 as seen from the user.

[0135] In both the example of the space-floating image display device employing the optical system of Figures 2A to 2C and the example of the space-floating image display device employing the optical system of Figure 2D, the space-floating image 3 is imaged in front of the transparent member 100, and the operation of the space-floating image 3 by the user's finger can be detected using the sensing light of the aerial operation detection sensor 1351 arranged on the back side of the transparent member 100 as seen from the user. Therefore, the space-floating image display device employing the optical system of Figure 2D has a different optical system from the space-floating image display device in which the optical system of Figures 2A to 2C is arranged on the back side of the transparent member 100 as seen from the user.

[0136] However, from the user's perspective, the usability of the space-floating image display device employing the optical system of FIG. 2D is almost the same as that of the space-floating image display device employing the optical system of FIGS. 2A to 2C.

[0137] Next, Fig. 4O is a diagram showing an example of the configuration of a space-floating image display device. Fig. 4O is a diagram showing the configuration of the internal optical system in the space-floating image display device 1000 of Fig. 4N. The space-floating image display device 1000 shown in Fig. 4O is equipped with an optical system corresponding to the optical system of Fig. 2D. The space-floating image display device 1000 shown in Fig. 4O is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward.

[0138] That is, in Fig. 4O, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. When the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230.

[0139] Here, the configuration of Fig. 4O is compared with the configuration of Fig. 4A to confirm the differences. In Fig. 4A, the display device 1 and the space-floating image 3 are in a plane-symmetrical relationship with respect to the plane of the polarization separation member 101. In contrast, in Fig. 4O, the display device 1 and the space-floating image 3 are in a plane-symmetrical relationship with respect to the plane of the retroreflector 5. Also, the configuration of Fig. 4A includes the retroreflector 2 and the λ / 4 plate 21, but these do not exist in Fig. 4O. Also, while it would be more preferable to include the absorbing polarizer 12 in Fig. 4A, the absorbing polarizer 12 is not particularly necessary in Fig. 4O.

[0140] That is, in order to replace the optical system of FIG. 2A in the configuration of FIG. 4A with the optical system of FIG. 2D and replace it with the configuration of FIG. 4O, the following can be done. That is, the polarization separation member 101 in the configuration of FIG. 4A can be replaced with the retroreflector 5, and the retroreflector 2 and the λ / 4 plate 21 can be removed from the configuration of FIG. 4A. The absorptive polarizer 12 can be omitted. If a replacement is made based on this idea, the optical system of FIG. 2A to FIG. 2C mounted on the configuration of the space floating image display device of FIG. 4A to FIG. 4G can be replaced with the optical system of FIG. 2D, and the space floating image display device mounted with the optical system of FIG. 2D can be replaced. At this time, the polarization separation member 101 can be replaced with the retroreflector 5 in FIG. 4A and FIG. 4B, and the polarization separation member 101B can be replaced with the retroreflector 5 in FIG. 4C to FIG. 4G.

[0141] In this way, a space-floating image display device can be realized in which the optical system in the configuration of the space-floating image display device in Figures 4A to 4G is replaced with the optical system in Figure 2D. Even in the space-floating image display device in which the optical system in Figure 2D is replaced, a space-floating image display device with almost the same usability as the space-floating image display device in Figures 4A to 4G can be realized.

[0142] <Display device> Next, the display device 1 of this embodiment will be described with reference to the drawings. The display device 1 of this embodiment includes an image display element 11 (liquid crystal display panel) and a light source device 13 that constitutes the light source thereof, and in Fig. 5, the light source device 13 is shown as an exploded perspective view together with the liquid crystal display panel.

[0143] As shown by the arrow 30 in Fig. 5, this liquid crystal display panel (image display element 11) receives an illumination light beam from a light source device 13, which is a backlight device, that has narrow-angle diffusion characteristics, i.e., has strong directivity (straightness) and characteristics similar to laser light with a polarization plane aligned in one direction. The liquid crystal display panel (image display element 11) modulates the received illumination light beam in accordance with an input video signal. The modulated image light is reflected by the retroreflector 2 and passes through a transparent member 100 to form a real image, which is a floating image in space (see Fig. 1).

[0144] In addition, in FIG. 5, the display device 1 is configured to include a liquid crystal display panel 11, a light direction conversion panel 54 that controls the directional characteristics of the light beam emitted from the light source device 13, and a narrow-angle diffusion plate (not shown) as necessary. That is, polarizing plates are provided on both sides of the liquid crystal display panel 11, and image light of a specific polarized wave is modulated in intensity by a video signal and emitted (see arrow 30 in FIG. 5). As a result, a desired image is projected as light of a specific polarized wave with high directivity (linearity) through the light direction conversion panel 54 toward the retroreflector 2, and after reflection by the retroreflector 2, it is transmitted toward the eyes of a monitor outside the store (space) to form a space-floating image 3. A protective cover 50 (see FIGS. 6 and 7) may be provided on the surface of the light direction conversion panel 54 described above.

[0145] <Display device example 1> FIG. 6 shows an example of a specific configuration of the display device 1. In FIG. 6, a liquid crystal display panel 11 and a light direction conversion panel 54 are arranged on the light source device 13 of FIG. 5. The light source device 13 is formed, for example, of plastic on the case shown in FIG. 5, and is configured by storing an LED element 201 and a light guide 203 inside. As shown in FIG. 5 and other figures, the end surface of the light guide 203 has a shape in which the cross-sectional area gradually increases toward the opposite side to the light receiving part in order to convert the divergent light from each LED element 201 into a substantially parallel light beam, and is provided with a lens shape that has an effect of gradually decreasing the divergence angle by multiple total reflections during propagation inside. The liquid crystal display panel 11 constituting the display device 1 is attached to the upper surface of the display device 1. In addition, an LED (Light Emitting Diode) element 201, which is a semiconductor light source, and an LED board 202 on which a control circuit for the LED element 201 is mounted are attached to one side surface of the case of the light source device 13 (the left end surface in this example). A heat sink, which is a member for cooling the heat generated by the LED elements and the control circuit, may be attached to the outer surface of the LED substrate 202.

[0146] In addition, the liquid crystal display panel frame (not shown) attached to the upper surface of the case of the light source device 13 is configured by attaching the liquid crystal display panel 11 attached to the frame, and further by attaching an FPC (Flexible Printed Circuits) (not shown) electrically connected to the liquid crystal display panel 11. That is, the liquid crystal display panel 11, which is an image display element, generates a display image by modulating the intensity of transmitted light based on a control signal from a control circuit (image control unit 1160 in FIG. 3) constituting an electronic device together with the LED element 201, which is a solid light source. At this time, the generated image light has a narrow diffusion angle and contains only specific polarization components, so that a new image display device that is similar to a surface-emitting laser image source driven by an image signal can be obtained. Note that, at present, it is technically and safety impossible to obtain a laser light beam of the same size as the image obtained by the above-mentioned display device 1 using a laser device. Therefore, in this embodiment, light similar to the above-mentioned surface-emitting laser image light is obtained from a light beam from a general light source equipped with an LED element, for example.

[0147] Next, the configuration of the optical system housed in the case of light source device 13 will be described in detail with reference to FIG. 7 together with FIG.

[0148] 6 and 7 are cross-sectional views, and only one of the LED elements 201 constituting the light source is shown, and this is converted into approximately collimated light by the shape of the light-receiving end surface 203a of the light guide 203. For this reason, the light-receiving part of the end surface of the light guide and the LED element are attached while maintaining a predetermined positional relationship.

[0149] Each of the light guides 203 is formed of a light-transmitting resin such as acrylic. The LED light receiving surface at the end of the light guide 203 has a cone-shaped outer circumferential surface obtained by rotating a parabolic cross section, and at the top of the light guide 203, a concave portion having a convex portion (i.e., a convex lens surface) formed in the center is formed, and at the center of the flat portion, a convex lens surface protruding outward (or a concave lens surface recessed inward) is formed (not shown). The outer shape of the light receiving portion of the light guide to which the LED element 201 is attached is a parabolic shape forming a cone-shaped outer circumferential surface, and is set within an angle range in which the light emitted from the LED element in the peripheral direction can be totally reflected inside the parabolic shape, or a reflective surface is formed.

[0150] On the other hand, the LED elements 201 are arranged at predetermined positions on the surface of the circuit board, that is, the LED substrate 202. The LED substrate 202 is arranged and fixed such that the LED elements 201 on the surface are positioned in the center of the recessed portion described above with respect to the LED collimator (light-receiving end surface 203a).

[0151] According to this configuration, the shape of light-receiving end surface 203a of light guide 203 makes it possible to extract the light emitted from LED element 201 as substantially parallel light, thereby improving the efficiency of use of the generated light.

[0152] As described above, the light source device 13 is configured by mounting a light source unit in which a plurality of LED elements 201 serving as light sources are arranged on the light receiving end surface 203a serving as a light receiving section provided on the end surface of the light guide 203, and the divergent light beam from the LED elements 201 is converted into substantially parallel light by the lens shape of the light receiving end surface 203a of the light guide end surface, and the light is guided inside the light guide 203 (in a direction parallel to the drawing) as shown by the arrow, and is emitted by the light beam direction conversion means 204 toward the liquid crystal display panel 11 arranged substantially parallel to the light guide 203 (in a direction perpendicular to the front side of the drawing). The uniformity of the light beam incident on the liquid crystal display panel 11 can be controlled by optimizing the distribution (density) of the light beam direction conversion means 204 depending on the shape of the inside or surface of the light guide.

[0153] The light beam direction conversion means 204 described above emits the light beam propagated inside the light guide toward the liquid crystal display panel 11 arranged substantially parallel to the light guide 203 (in a direction perpendicular to the front of the drawing) by changing the shape of the light guide surface or by providing, for example, a portion with a different refractive index inside the light guide. At this time, when the brightness of the screen center and the screen periphery are compared while facing the liquid crystal display panel 11 directly at the center of the screen and placing the viewpoint at the same position as the diagonal dimension of the screen, if the relative brightness ratio is 20% or more, there is no practical problem, and if it exceeds 30%, it is an even better characteristic.

[0154] 6 is a cross-sectional layout diagram for explaining the configuration and operation of a light source of this embodiment that performs polarization conversion in light source device 13 including the above-mentioned light guide 203 and LED element 201. In Fig. 6, light source device 13 is composed of light guide 203 formed of, for example, plastic or the like and provided with light beam direction conversion means 204 on its surface or inside, LED element 201 as a light source, reflective sheet 205, retardation plate 206, lenticular lens, etc., and on the upper surface thereof is attached liquid crystal display panel 11 equipped with polarizing plates on the light source light entrance surface and image light exit surface.

[0155] A film or sheet-like reflective polarizing plate 49 is provided on the light source light incidence surface (lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, and selectively reflects one side of the polarized wave (e.g. P wave) 212 of the natural light beam 210 emitted from the LED element 201. The reflected light is reflected again by a reflecting sheet 205 provided on one surface (lower surface in the figure) of the light guide 203, and directed toward the liquid crystal display panel 11. Therefore, a retardation plate (λ / 4 plate) is provided between the reflecting sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49, and the reflected light beam is reflected by the reflecting sheet 205 and passes through twice to convert the reflected light beam from P polarized light to S polarized light, improving the utilization efficiency of the light source light as image light. The image light beam, the light intensity of which is modulated by the image signal in the liquid crystal display panel 11 (arrow 213 in FIG. 6), enters the retroreflector 2. After reflection by the retroreflector 2, a real image, a floating image in space, can be obtained.

[0156] Fig. 7 is a cross-sectional layout diagram for explaining the configuration and action of a light source of this embodiment that performs polarization conversion in light source device 13 including light guide 203 and LED element 201, similar to Fig. 6. Light source device 13 is also similarly composed of light guide 203 formed of, for example, plastic, on the surface or inside of which light flux direction conversion means 204 is provided, LED element 201 as a light source, reflection sheet 205, retardation plate 206, lenticular lens, etc. On the upper surface of light source device 13, liquid crystal display panel 11 having polarizing plates on the light source light entrance surface and the image light exit surface is attached as an image display element.

[0157] A film or sheet-like reflective polarizing plate 49 is provided on the light source light incidence surface (lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, and selectively reflects one side of the polarized wave (e.g., S wave) 211 of the natural light beam 210 emitted from the LED element 201. That is, in the example of FIG. 7, the selective reflection characteristic of the reflective polarizing plate 49 is different from that of FIG. 7. The reflected light is reflected by the reflective sheet 205 provided on one surface (lower surface in the figure) of the light guide 203, and again heads toward the liquid crystal display panel 11. A retardation plate (λ / 4 plate) is provided between the reflective sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49, and the reflected light beam is reflected by the reflective sheet 205 and passes through it twice, thereby converting the reflected light beam from S polarized light to P polarized light, and improving the utilization efficiency of the light source light as image light. The image light beam intensity-modulated by the image signal in the liquid crystal display panel 11 (arrow 214 in FIG. 7) enters the retroreflector 2. After reflection by the retroreflector 2, a real image, a floating image in space, can be obtained.

[0158] In the light source devices shown in Figures 6 and 7, in addition to the action of the polarizing plate provided on the light incidence surface of the corresponding liquid crystal display panel 11, the polarized component on one side is reflected by the reflective polarizing plate, so that the theoretically obtainable contrast ratio is the reciprocal of the cross transmittance of the reflective polarizing plate multiplied by the reciprocal of the cross transmittance obtained by the two polarizing plates attached to the liquid crystal display panel. This results in high contrast performance. In fact, it was confirmed by experiments that the contrast performance of the displayed image was improved by more than 10 times. As a result, a high-quality image comparable to that of a self-luminous organic EL was obtained.

[0159] <Display device example 2> 8 shows another example of a specific configuration of the display device 1. This light source device 13 is configured by housing LEDs, a collimator, a composite diffusion block, a light guide, etc., in a case made of, for example, plastic, and has a liquid crystal display panel 11 attached to its upper surface. LED (Light Emitting Diode) elements 14a and 14b, which are semiconductor light sources, and an LED board mounting a control circuit for the LED elements are attached to one side of the case of the light source device 13, and a heat sink 103, which is a member for cooling heat generated by the LED elements and the control circuit, is attached to the outer side of the LED board.

[0160] The liquid crystal display panel frame attached to the top surface of the case is configured to have the liquid crystal display panel 11 attached to the frame, and further, an FPC (Flexible Printed Circuits) 403 electrically connected to the liquid crystal display panel 11, etc. attached to it. That is, the liquid crystal display panel 11, which is a liquid crystal display element, generates a display image by modulating the intensity of transmitted light together with the LED elements 14a and 14b, which are solid-state light sources, based on a control signal from a control circuit (not shown here) that constitutes the electronic device.

[0161] <Display device example 3> Next, another example of the specific configuration of the display device 1 (Example 3 of the display device) will be described with reference to Fig. 9. The light source device of this display device 1 converts the divergent light flux of the light (a mixture of P-polarized light and S-polarized light) from the LED into a substantially parallel light flux by the collimator 18, and reflects the light toward the liquid crystal display panel 11 by the reflecting surface of the reflective light guide 304. The reflected light is incident on the reflective polarizing plate 49 disposed between the liquid crystal display panel 11 and the reflective light guide 304. The reflective polarizing plate 49 transmits light of a specific polarized wave (e.g., P-polarized light) and causes the transmitted polarized light to be incident on the liquid crystal display panel 11. Here, polarized waves other than the specific polarized wave (e.g., S-polarized light) are reflected by the reflective polarizing plate 49 and head toward the reflective light guide 304 again.

[0162] The reflective polarizing plate 49 is disposed at an angle with respect to the liquid crystal display panel 11 so as not to be perpendicular to the principal ray of the light from the reflecting surface of the reflective light guide 304. The principal ray of the light reflected by the reflective polarizing plate 49 is incident on the transmitting surface of the reflective light guide 304. The light incident on the transmitting surface of the reflective light guide 304 is transmitted through the rear surface of the reflective light guide 304, transmitted through the λ / 4 plate 270 which is a retardation plate, and reflected by the reflecting plate 271. The light reflected by the reflecting plate 271 is transmitted through the λ / 4 plate 270 again, and transmitted through the transmitting surface of the reflective light guide 304. The light transmitted through the transmitting surface of the reflective light guide 304 is incident on the reflective polarizing plate 49 again.

[0163] At this time, the light re-entering the reflective polarizing plate 49 has passed through the λ / 4 plate 270 twice, and therefore its polarization has been converted to a polarized wave (for example, P-polarized light) that is transmitted through the reflective polarizing plate 49. Therefore, the light whose polarization has been converted is transmitted through the reflective polarizing plate 49 and enters the liquid crystal display panel 11. Note that, with regard to the polarization design related to the polarization conversion, the polarization may be configured inversely from the above explanation (S-polarized light and P-polarized light may be reversed).

[0164] As a result, the light from the LED is aligned to a specific polarization (e.g., P polarization), enters the liquid crystal display panel 11, and is brightness-modulated according to the video signal to display an image on the panel surface. As in the above example, multiple LEDs that make up the light source are shown (however, because this is a vertical cross section, only one is shown in FIG. 9), and these are attached at predetermined positions relative to the collimator 18.

[0165] The collimators 18 are each formed of, for example, a light-transmitting resin such as acrylic or glass. The collimators 18 may have a cone-shaped outer peripheral surface obtained by rotating a parabolic cross section. The collimators 18 may have a concave portion with a convex portion (i.e., a convex lens surface) formed in the center of the apex (the side facing the LED substrate 102). The collimators 18 have a convex lens surface protruding outward (or a concave lens surface recessed inward) in the center of the flat surface (the side opposite to the apex). The parabolic surface forming the cone-shaped outer peripheral surface of the collimators 18 is set within an angle range that allows the light emitted from the LED in the peripheral direction to be totally reflected therein, or a reflective surface is formed.

[0166] The LEDs are arranged at predetermined positions on the surface of the circuit board, that is, the LED board 102. The LED board 102 is arranged and fixed to the collimator 18 so that the LEDs on the surface are located at the center of the apex of the convex cone shape (or in the concave portion if the apex has a concave portion).

[0167] According to this configuration, among the light emitted from the LED by the collimator 18, the light emitted from the central part is condensed into parallel light by the convex lens surface forming the outer shape of the collimator 18. The light emitted from the other parts toward the periphery is reflected by the parabolic surface forming the outer peripheral surface of the cone shape of the collimator 18, and is condensed into parallel light in the same manner. In other words, according to the collimator 18 having a convex lens in its center and a parabolic surface formed on its periphery, it is possible to extract almost all of the light generated by the LED as parallel light, and it is possible to improve the utilization efficiency of the generated light.

[0168] Furthermore, the light converted into approximately parallel light by the collimator 18 shown in FIG. 9 is reflected by the reflective light guide 304. The light of a specific polarized wave is transmitted through the reflective polarizing plate 49 by the action of the reflective polarizing plate 49, and the light of the other polarized wave reflected by the action of the reflective polarizing plate 49 is transmitted through the light guide 304 again. The light is reflected by the reflector 271 located opposite the liquid crystal display panel 11 with respect to the reflective light guide 304. At this time, the light is polarized and converted by passing twice through the λ / 4 plate 270, which is a retardation plate. The light reflected by the reflector 271 is transmitted through the light guide 304 again and enters the reflective polarizing plate 49 provided on the opposite surface. Since the incident light has been polarized and converted, it is transmitted through the reflective polarizing plate 49 and enters the liquid crystal display panel 11 with the same polarization direction. As a result, all the light from the light source can be used, and the geometrical optical utilization efficiency of light is doubled. In addition, since the degree of polarization (extinction ratio) of the reflective polarizer is also included in the extinction ratio of the entire system, the use of the light source device of this embodiment significantly improves the contrast ratio of the entire display device. By adjusting the surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflector 271, the reflection diffusion angle of light at each reflective surface can be adjusted. The surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflector 271 can be adjusted for each design so that the uniformity of the light incident on the liquid crystal display panel 11 is more suitable.

[0169] It should be noted that the λ / 4 plate 270, which is the retardation plate in Fig. 9, does not necessarily have to have a phase difference of λ / 4 with respect to polarized light that is perpendicularly incident on the λ / 4 plate 270. In the configuration of Fig. 9, any retardation plate may be used as long as the phase changes by 90° (λ / 2) when polarized light passes through it twice. The thickness of the retardation plate may be adjusted according to the incidence angle distribution of the polarized light.

[0170] <Display device example 4> Furthermore, another example (display example 4) of the configuration of the optical system such as the light source device of the display device will be described with reference to Fig. 10. This is a configuration example in which a diffusion sheet is used instead of the reflective light guide 304 in the light source device of display example 3. Specifically, two optical sheets (optical sheet 207A and optical sheet 207B) that convert the diffusion characteristics in the vertical and horizontal directions (front and back directions not shown in the figure) of the drawing are used on the light emission side of the collimator 18, and the light from the collimator 18 is made to enter between the two optical sheets (diffusion sheets).

[0171] The optical sheet may be one sheet instead of two sheets. In the case of one sheet, the vertical and horizontal diffusion characteristics are adjusted by the fine shape of the front and back surfaces of the single optical sheet. In addition, a plurality of diffusion sheets may be used to share the functions. Here, in the example of FIG. 10, the reflection diffusion characteristics due to the front and back shapes of the optical sheets 207A and 207B may be optimally designed with the number of LEDs, the divergence angle from the LED substrate (optical element) 102, and the optical specifications of the collimator 18 as design parameters so that the surface density of the light beam emitted from the liquid crystal display panel 11 is uniform. In other words, the diffusion characteristics are adjusted by the surface shapes of a plurality of diffusion sheets instead of the light guide.

[0172] In the example of FIG. 10, the polarization conversion is performed in the same manner as in the display device example 3 described above. That is, in the example of FIG. 10, the reflective polarizing plate 49 may be configured to have a property of reflecting S-polarized light (transmitting P-polarized light). In this case, the reflective polarizing plate 49 transmits P-polarized light out of the light emitted from the LED light source, and the transmitted light enters the liquid crystal display panel 11. The reflective polarizing plate 49 reflects S-polarized light out of the light emitted from the LED light source, and the reflected light passes through the retardation plate 270 shown in FIG. 10. The light that passes through the retardation plate 270 is reflected by the reflector 271. The light reflected by the reflector 271 passes through the retardation plate 270 again and is converted to P-polarized light. The polarization-converted light passes through the reflective polarizing plate 49 and enters the liquid crystal display panel 11.

[0173] It should be noted that the λ / 4 plate 270, which is the retardation plate in Fig. 10, does not necessarily have to have a phase difference of λ / 4 with respect to polarized light perpendicularly incident on the λ / 4 plate 270. In the configuration of Fig. 10, any retardation plate may be used in which the phase changes by 90° (λ / 2) when polarized light passes through it twice. The thickness of the retardation plate may be adjusted according to the distribution of the incident angles of the polarized light. It should be noted that, in Fig. 10 as well, the polarization design related to the polarization conversion may be configured in reverse (reversing the S-polarized light and the P-polarized light) based on the above explanation.

[0174] In a device for general TV use, the light emitted from the liquid crystal display panel 11 has similar diffusion characteristics in both the horizontal direction of the screen (shown by the X-axis in FIG. 12(a)) and the vertical direction of the screen (shown by the Y-axis in FIG. 12(b)). In contrast, the diffusion characteristics of the light flux emitted from the liquid crystal display panel of this embodiment are 1 / 5 of the 62 degrees of a device for general TV use, when the viewing angle at which the luminance is 50% of that when viewed from the front (angle 0 degrees) is set to 13 degrees, as shown in Example 1 of FIG. 12. Similarly, the vertical viewing angle is optimized by optimizing the reflection angle of the reflective light guide and the area of ​​the reflection surface so that the viewing angle on the upper side is unequal between the top and bottom and is kept to about 1 / 3 of the viewing angle on the lower side. As a result, the amount of image light heading in the monitoring direction is significantly improved compared to conventional liquid crystal TVs, and the luminance is 50 times or more higher.

[0175] Furthermore, assuming the viewing angle characteristics shown in Example 2 in Figure 12, the viewing angle at which the brightness is 50% of that when viewed from the front (angle 0 degrees) is set to 5 degrees, which is 1 / 12 of the 62 degrees of devices used for general TV applications. Similarly, the vertical viewing angle is optimized by optimizing the reflection angle of the reflective light guide and the area of ​​the reflective surface so that the viewing angle is approximately 1 / 12 of that of devices used for general TV applications, with equal viewing angles both above and below. As a result, the amount of image light heading in the monitoring direction is significantly improved compared to conventional LCD TVs, and the brightness is more than 100 times higher.

[0176] As described above, by setting the viewing angle to a narrow angle, the amount of luminous flux heading in the monitoring direction can be concentrated, greatly improving the efficiency of light utilization. As a result, even if a liquid crystal display panel for general TV applications is used, by controlling the light diffusion characteristics of the light source device, it is possible to achieve a significant improvement in brightness with similar power consumption, making it possible to create an image display device that is compatible with information display systems facing bright outdoors.

[0177] When using a large LCD panel, the overall brightness of the screen can be improved by directing the light around the screen inward so that it is directed toward the observer when the observer faces the center of the screen. Figure 11 shows the convergence angle of the long and short sides of the panel when the observer's distance from the panel L and the panel size (screen ratio 16:10) are used as parameters. When monitoring with the screen vertically long, the convergence angle can be set according to the short side. For example, when using a 22-inch panel vertically and the monitoring distance is 0.8 m, a convergence angle of 10 degrees will allow the image light from the four corners of the screen to be effectively directed toward the observer.

[0178] Similarly, when monitoring with a 15-inch panel in portrait orientation and a monitoring distance of 0.8 m, a convergence angle of 7 degrees will allow the image light from the four corners of the screen to be effectively directed toward the observer. As described above, depending on the size of the LCD panel and whether it is used portrait or landscape, the overall brightness of the screen can be improved by directing the image light from the periphery of the screen toward the observer who is in the optimal position to monitor the center of the screen.

[0179] The basic configuration, as shown in FIG. 9, is that a light beam with a narrow-angle directional characteristic is incident on a liquid crystal display panel 11 from a light source device, and the brightness is modulated according to a video signal, so that the video information displayed on the screen of the liquid crystal display panel 11 is reflected by a retroreflector, and the resulting floating-in-space image is displayed indoors or outdoors via a transparent member 100.

[0180] By using the display device and light source device according to one embodiment of the present invention described above, it is possible to realize a space floating image display device with higher light utilization efficiency.

[0181] <Example of image display processing in a space floating image display device> Next, an example of a problem solved by the image processing of this embodiment will be described with reference to Fig. 13A. In the space-floating image display device 1000, when the back side of the space-floating image 3 from the user's perspective is inside the housing of the space-floating image display device 1000 and is sufficiently dark, the user visually recognizes that the background of the space-floating image 3 is black.

[0182] Here, an example of displaying a character "panda" 1525 in the space floating image 3 will be described with reference to Fig. 13A. First, the image control unit 1160 in Fig. 3 distinguishes and recognizes the pixel area in which the image of the character "panda" 1525 is drawn from the transparent information area 1520 which is the background image, for an image including the pixel area in which the image of the character "panda" 1525 is drawn and the transparent information area 1520 which is the background image, as shown in Fig. 13A(1).

[0183] A method for distinguishing and recognizing a character image from a background image may, for example, be configured so that the image processing of the video control unit 1160 can process the background image layer and the layer of a character image in front of the background image layer as separate layers, and the character image and background image can be distinguished and recognized based on the overlay relationship when these layers are combined.

[0184] Here, the image control unit 1160 recognizes black pixels that draw objects such as character images and transparent information pixels as different information. However, it is assumed that the brightness of both the black pixels that draw objects and the transparent information pixels is 0. In this case, when the space floating image 3 is displayed, there is no difference in brightness between the pixels that draw black in the image of the character "panda" 1525 and the pixels of the transparent information area 1520, which is the background image. Therefore, in the space floating image 3, as shown in FIG. 13A(2), there is no brightness in either the pixels that draw black in the image of the character "panda" 1525 and the pixels of the transparent information area 1520, and they are visually recognized by the user as optically the same black space. In other words, the part that draws black in the image of the character "panda" 1525, which is an object, blends into the background, and only the non-black part of the character "panda" 1525 is recognized as an image floating in the display area of ​​the space floating image 3.

[0185] An example of image processing of this embodiment will be described with reference to FIG. 13B. FIG. 13B is a diagram for explaining an example of image processing that more suitably solves the problem that the black image area of ​​the object blends into the background, as described in FIG. 13A. In FIG. 13B(1) and (2), the upper side shows the display state of the floating in space image 3, and the lower side shows the input / output characteristics of the image processing of the image of the object. Note that the image of the object (character "panda" 1525) and the corresponding data may be read from the storage unit 1170 or memory 1109 in FIG. 3. Alternatively, they may be input from the image signal input unit 1131. Alternatively, they may be acquired via the communication unit 1132.

[0186] Here, in the state of Fig. 13B(1), the input / output characteristics of the image processing of the image of the object are in a linear state with no particular adjustment. In this case, the display state is the same as that of Fig. 13A(2), and the black image area of ​​the object blends into the background. In contrast, in Fig. 13B(2), the video control unit 1160 of this embodiment adjusts the input / output characteristics of the image processing of the image of the object (character "panda" 1525) to the input / output characteristics shown in the lower part.

[0187] That is, the video control unit 1160 performs image processing of input / output characteristics on the image of the object (character "panda" 1525) that has a characteristic of converting the pixels of the input image into output pixels with increased luminance values ​​of pixels in low luminance areas. After the image of the object (character "panda" 1525) is subjected to the image processing of the input / output characteristics, a video including the image of the object (character "panda" 1525) is input to the display device 1 and displayed. Then, as shown in the upper part of FIG. 13B(2), the display state of the floating in space video 3 is such that the luminance of pixel areas that draw black in the image of the character "panda" 1525 increases. As a result, the user can recognize the area that draws black among the areas that draw the image of the character "panda" 1525 as distinct from the black of the background without blending in, and the object can be displayed more suitably.

[0188] That is, by using the image processing of Fig. 13B(2), the area displaying the image of the character "panda" 1525, which is an object, can be distinguished from the black background inside the housing of the space-floating image display device 1000 through the window, improving the visibility of the object. Therefore, for example, even if the pixels constituting the object include pixels with a brightness value of 0 before the image processing (i.e., the time when the image of the object and the corresponding data are read from the storage unit 1170 or memory 1109 in Fig. 3, the time when the image of the object is input from the video signal input unit 1131, or the time when the data of the object is obtained via the communication unit 1132, etc.), the image is converted into an object with a high brightness value of the pixels in the low brightness area by the image processing of the input / output characteristics by the video control unit 1160, and then displayed on the display device 1, and is converted into the space-floating image 3 by the optical system of the space-floating image display device 1000.

[0189] In other words, the pixels that make up the object after image processing of the input / output characteristics are converted to a state in which they do not include pixels with a brightness value of 0, and then the object is displayed on the display device 1 and converted into a floating-in-space image 3 by the optical system of the floating-in-space image display device 1000.

[0190] In the image processing of FIG. 13B(2), a method for applying image processing with the input / output characteristics of FIG. 13B(2) only to the image area of ​​the object (character “panda” 1525) is, for example, to configure the image processing of the video control unit 1160 so that the background image layer and the layer of the character image in front of the background image layer can be processed as separate layers, and the image processing with the input / output characteristics of FIG. 13B(2) is applied to the layer of the character image while not applying the image processing to the background image layer.

[0191] Then, by compositing these layers, image processing with a characteristic of raising the low brightness area of ​​the input image is performed only on the character image, as shown in Fig. 13B(2). Alternatively, after the character image layer and the background image layer are composited, the image processing with the input / output characteristic of Fig. 13B(2) may be performed only on the character image area.

[0192] In addition, the input / output image characteristics used in the image processing for boosting the low luminance region of the input / output characteristics for the input image are not limited to the example of FIG. 13B(2). Any image processing for boosting low luminance may be used, including so-called brightness adjustment. Alternatively, image processing for improving visibility may be performed by controlling the gain that changes the weighting of the Retinex processing, as disclosed in International Publication WO 2014 / 162533.

[0193] According to the image processing of FIG. 13B(2) described above, it is possible to allow the user to recognize areas in which black is drawn among areas in which images of characters, objects, etc. are drawn without blending into the black background, thereby making it possible to realize a more suitable display.

[0194] In the example of Figures 13A and 13B, the problem and more suitable image processing are explained using a space-floating image display device in which the background appears black (for example, the space-floating image display device 1000 in Figures 4A to 4G, or the space-floating image display device 1000 in the state in which the rear window is blocked in Figures 4I and 4J). However, the image processing is also effective in devices other than these space-floating image display devices.

[0195] Specifically, in the space-floating image display device 1000 in Fig. 4H, or in Fig. 4I and Fig. 4J where the rear window is not shaded, the background of the space-floating image 3 is not black, but the view behind the space-floating image display device 1000 through the window. In this case, the problems described in Fig. 13A and Fig. 13B also exist.

[0196] That is, the part of the image of the character "panda" 1525, which is an object, that is drawn in black will blend into the scenery behind the space-floating image display device 1000 through the window. In this case, by using the image processing of Fig. 13B(2), the part of the image of the character "panda" 1525, which is an object, that is drawn in black will be distinguished from the scenery behind the space-floating image display device 1000 through the window, improving the visibility of the object.

[0197] In other words, by using the image processing of FIG. 13B(2), the area displaying the image of the object character “panda” 1525 can be recognized as distinct from the scenery behind the space floating image display device 1000 through the window, and it can be more clearly recognized that the object character “panda” 1525 is in front of the scenery, improving the visibility of the object.

[0198] In addition, in the space-floating image display device 1000 of Figures 4K, 4L, and 4M, as described above, when another image (such as an image of the transmissive self-luminous image display device 1650 or an image of the second display device 1680) is displayed at a position different in depth from the space-floating image 3, the background of the space-floating image 3 is not black, but the other image. In this case, the problem described in Figures 13A and 13B also exists.

[0199] That is, the part of the image of the character "panda" 1525, which is an object, that is drawn in black will blend into the other image that is displayed at a different depth from the floating in space image 3. In this case as well, by using the image processing of Fig. 13B(2), the part of the image of the character "panda" 1525, which is an object, that is drawn in black can be recognized as being distinct from the other image, improving the visibility of the object.

[0200] In other words, by using the image processing of Figure 13B (2), the area displaying the image of the object character "panda" 1525 can be recognized as distinct from the other image, and it can be more easily recognized that the object character "panda" 1525 is in front of the other image, improving the visibility of the object.

[0201] An example of the image display process of this embodiment will be described with reference to Fig. 13C. Fig. 13C shows an example of the image display of this embodiment in which the floating-in-space image 3 and a second image 2050, which is another image, are displayed simultaneously. The second image 2050 may correspond to the display image of the transmissive self-luminous image display device 1650 of Fig. 4K or Fig. 4L. The second image 2050 may also correspond to the display image of the second display device 1680 of Fig. 4M.

[0202] That is, the example of the image display in Fig. 13C shows an example of a specific example of the image display example of the space-floating image display device 1000 in Fig. 4K, Fig. 4L, and Fig. 4M. In the example of this figure, a bear character is displayed in the space-floating image 3. The area other than the bear character in the space-floating image 3 is displayed in black, and becomes transparent as a space-floating image. Also, the second image 2050 is a background image in which a plain, a mountain, and a sun are drawn.

[0203] Here, in Fig. 13C, the floating-in-space image 3 and the second image 2050 are displayed at different depth positions. When the user 230 views the two images, the floating-in-space image 3 and the second image 2050, in the line of sight of the arrow 2040, the user 230 can view the images in a state where the two images are superimposed. Specifically, the bear character of the floating-in-space image 3 appears superimposed in front of the background of the plains, mountains, and sun depicted in the second image 2050.

[0204] Here, since the floating-in-space image 3 is formed as a real image in the air, when the user 230 moves his / her viewpoint slightly, the user 230 can recognize the depth of the floating-in-space image 3 and the second image 2050 due to parallax. Therefore, the user 230 can get a stronger sense of floating in space from the floating-in-space image 3 while viewing the two images in an overlapping state.

[0205] An example of the image display process of this embodiment will be described with reference to Fig. 13D. Fig. 13D(1) is a diagram showing the floating in space image 3 from the line of sight of the user 230, among the image display examples of this embodiment of Fig. 13C. Here, a bear character is displayed in the floating in space image 3. The area other than the bear character in the floating in space image 3 is displayed in black, and becomes transparent as a floating in space image.

[0206] Fig. 13D(2) is a diagram showing the second image 2050 in the example of the image display of this embodiment in Fig. 13C as viewed from the line of sight of the user 230. In the example of this figure, the second image 2050 is a background image in which a plain, a mountain, and the sun are drawn.

[0207] Fig. 13D(3) is a diagram showing a state in which the second image 2050 and the floating in space image 3 are seen superimposed in the line of sight of the user 230, among the example of image display of this embodiment in Fig. 13C. Specifically, the bear character of the floating in space image 3 is seen superimposed in front of the background of the plains, mountains, and sun depicted in the second image 2050.

[0208] Here, when the space-floating image 3 and the second image 2050 are displayed simultaneously, it is desirable to pay attention to the balance of brightness between the two images in order to more suitably ensure the visibility of the space-floating image 3. If the second image 2050 is too bright compared to the brightness of the space-floating image 3, the display image of the space-floating image 3 will be transparent, and the second image 2050, which is the background, will be strongly visible through the transmitted image.

[0209] Therefore, the light source output of the spatially floating image 3 and the display image luminance of the display device 1, and the light source output of the display device displaying the second image 2050 and the display image luminance of the display device should be set so that at least the brightness per unit area of ​​the spatially floating image 3 at the display position of the spatially floating image 3 is greater than the brightness per unit area of ​​the image light reaching the display position of the spatially floating image 3 from the second image 2050.

[0210] In addition, since this condition only needs to be satisfied when the space-floating image 3 and the second image 2050 are displayed simultaneously, when switching from the first display mode in which the space-floating image 3 is not displayed and only the second image 2050 is displayed to the second display mode in which the space-floating image 3 and the second image 2050 are displayed simultaneously, control may be performed to reduce the brightness of the second image 2050 by lowering the output of the light source of the display device that displays the second image 2050 and / or the display image luminance of the display device. These controls may be realized by the control unit 1110 in Fig. 3 controlling the display device 1 and the display device that displays the second image 2050 (the transmissive self-luminous image display device 1650 in Fig. 4K or Fig. 4L or the second display device 1680 in Fig. 4M).

[0211] In addition, when controlling to reduce the brightness of the second image 2050 in switching from the above-mentioned first display mode to the above-mentioned second display mode, the brightness may be reduced uniformly across the entire screen of the second image 2050. Alternatively, instead of reducing the brightness uniformly across the entire screen of the second image 2050, the brightness reduction effect may be maximized in the portion where the object is displayed in the floating-in-space image 3, and the brightness reduction effect may be gradually relaxed around the portion. In other words, if the brightness reduction of the second image 2050 is realized only in the portion where the floating-in-space image 3 is superimposed on the second image 2050 and viewed, the visibility of the floating-in-space image 3 is sufficiently ensured.

[0212] Here, since the floating-in-space image 3 and the second image 2050 are displayed at positions with different depths, when the user 230 slightly changes his / her viewpoint, the superimposed position of the floating-in-space image 3 on the second image 2050 changes due to parallax. Therefore, when reducing the brightness unevenly across the entire screen of the second image 2050 in switching from the first display mode to the second display mode, it is not desirable to reduce the brightness sharply based on the contour of the object displayed in the floating-in-space image 3, but it is desirable to perform a gradation process of the brightness reduction effect, which changes the brightness reduction effect stepwise depending on the position as described above.

[0213] In addition, in the space-floating image display device 1000 where the position of the object displayed in the space-floating image 3 is approximately at the center of the space-floating image 3, the position where the brightness reduction effect of the gradation processing of the brightness reduction effect is the highest can be set to the center position of the space-floating image 3.

[0214] According to the image display process of this embodiment described above, the user 230 can visually recognize the space floating image 3 and the second image 2050 more favorably.

[0215] Note that when displaying the space-floating image 3, control may be performed so as not to display the second image 2050. Since the visibility of the space-floating image 3 is improved when the second image 2050 is not displayed, this is suitable for the space-floating image display device 1000 and the like in which the user must be able to reliably view the space-floating image 3 when the space-floating image 3 is displayed.

[0216] <Example 2> As the second embodiment of the present invention, an example of another configuration example of the space-floating image display device will be described. Note that the space-floating image display device according to this embodiment is obtained by changing the optical system stored in the space-floating image display device described in the first embodiment to the optical system shown in FIG. 14(1) or FIG. 14(2). In this embodiment, differences from the first embodiment will be described, and repeated explanations of the same configuration as the first embodiment will be omitted. Note that in the following description of this embodiment, the predetermined polarized light and the other polarized light are polarized waves whose phases differ from each other by 90°.

[0217] Fig. 14(1) is an example of an optical system and an optical path according to this embodiment. The optical system shown in Fig. 14(1) is configured to be more compact as a whole optical system by moving the display device 1 closer to the polarization separation member 101B in the optical system of Fig. 2C. In Fig. 14(1), the configurations with the same reference numerals as Fig. 2C are omitted from the repeated detailed description.

[0218] 14(1), similar to Fig. 2C, image light of a predetermined polarized light (P polarized light in the figure) emitted from display device 1 travels in a direction perpendicular to the image display surface of display device 1. Here, similar to Fig. 2C, polarization separation member 101B selectively transmits the predetermined polarized light (P polarized light in the figure) emitted from display device 1 and reflects the other polarized light (S polarized light in the figure).

[0219] Therefore, the image light of a predetermined polarization (P-polarized light in the figure) traveling in the vertical direction from the image display surface of the display device 1 passes through the polarization separation member 101B and reaches the retroreflector 2 to which the λ / 4 plate 21 is attached. The image light that is retroreflected by the retroreflector 2 and travels again toward the polarization separation member 101B is converted from the predetermined polarization (P-polarized light in the figure) at the time of emission from the display device 1 to the other polarization (S-polarized light in the figure) by passing through the λ / 4 plate 21 twice. The image light that travels again toward the polarization separation member 101B is the other polarization (S-polarized light in the figure), and is therefore reflected by the polarization separation member 101B toward the position where the user should be. The traveling direction of the image reflected by the polarization separation member 101B is determined based on the angle at which the polarization separation member 101B is arranged.

[0220] In the example of FIG. 14(1), the image light traveling toward the polarization separation member 101B is reflected at a right angle by the polarization separation member 101B and travels as shown in the figure. The image light reflected by the polarization separation member 101B forms a space-floating image 3A. The space-floating image 3A can be visually recognized by the user from the direction of the arrow A.

[0221] Here, due to the characteristics of retroreflection by the retroreflector 2, the optical path length of the image light emitted from the display device 1 until it reaches the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 until it reaches the formation position of the space floating image 3A. This relationship determines the formation position of the space floating image 3A in the traveling direction of the image light reflected by the polarization separation member 101B.

[0222] In the example of FIG. 14(1), the display device 1, the polarization separation member 101B, and the retroreflector 2 are arranged closer than in the example of FIG. 2C. This allows the entire optical system to be configured more compactly. However, the amount of the space-floating image 3A projecting from the optical system of FIG. 14(1) is not very large. For example, as an index of the amount of the space-floating image 3A projecting from the optical system, the distance from the position where the central light beam of the image light is reflected by the polarization separation member 101B to the position where the image light forms the space-floating image 3A is shown in the figure (L1 in the example of FIG. 14(1)).

[0223] In addition, in the polarization design of the optical system in Fig. 14(1), the characteristics of P polarization and S polarization may be interchanged. Specifically, a predetermined polarization of the image light emitted from the display device 1 may be S polarization, and the characteristics of P polarization and S polarization may be interchanged in the reflection characteristics of the polarization separation member 101B. In this case, the illustrated P polarization and S polarization are both reversed, but the optical design such as the optical path can be realized in exactly the same way.

[0224] Next, Fig. 14(2) shows another example of the optical system and the optical path according to this embodiment. The optical system in Fig. 14(2) is a modified version of the optical system in Fig. 14(1) in order to increase the amount of the floating image projected from the optical system while realizing the same compactness as the optical system in Fig. 14(1). In Fig. 14(2), the detailed description of the components with the same reference numerals as those in Fig. 14(1) will be omitted.

[0225] 14(2), similarly to FIG. 14(1), image light of a predetermined polarized light (P polarized light in the figure) emitted from display device 1 travels in a vertical direction from the image display surface of display device 1. Here, the polarization characteristics of polarization separation member 101B are arranged 90 degrees differently from FIG. 14(1). Image light of a predetermined polarized light (P polarized light in the figure) traveling in a vertical direction from the image display surface of display device 1 passes through polarization separation member 101B.

[0226] 14(1), a specular reflector 4 to which a λ / 4 plate 21B is attached is disposed after the image light passes through the polarization separation member 101B, rather than a retroreflector 2 to which a λ / 4 plate 21 is attached. Here, the reflection at the specular reflector 4 is specular reflection (also called regular reflection), not retroreflection.

[0227] Therefore, the image light that has passed through the polarization separation member 101B is specularly reflected by the specular reflector 4 to which the λ / 4 plate 21B is attached. The image light that has been specularly reflected by the specular reflector 4 and travels again toward the polarization separation member 101B has been converted from the specified polarization (P-polarized in the figure) at the time of emission from the display device 1 to the other polarization (S-polarized in the figure) by having passed twice through the λ / 4 plate 21. The image light that has traveled again toward the polarization separation member 101B is the other polarization (S-polarized in the figure), and is therefore reflected by the polarization separation member 101B.

[0228] Here, since the orientation of the polarization separation member 101B in FIG. 14(2) is different from that in FIG. 14(1), the image light reflected by the polarization separation member 101B travels in the opposite direction to the position where the user should be. A retroreflector 2 with a λ / 4 plate 21C attached is disposed at the destination of the image light reflected by the polarization separation member 101B. The image light is retroreflected by the retroreflector 2. The image light that is retroreflected by the retroreflector 2 and travels again toward the polarization separation member 101B has been converted from the other polarized light (S polarized light in the figure) to the specified polarized light (P polarized light in the figure) again by transmitting twice through the λ / 4 plate 21C.

[0229] The image light that travels back toward the polarization separation member 101B is a predetermined polarization (P polarization in the figure), so it passes through the polarization separation member 101B and travels directly toward the position where the user should be. The image light that passes through the polarization separation member 101B forms the space floating image 3B. The space floating image 3B can be visually recognized by the user from the direction of the arrow A.

[0230] 14(2), as in FIG. 14(1), due to the characteristics of retroreflection by the retroreflector 2, the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 to reach the formation position of the space floating image 3B. This relationship determines the formation position of the space floating image 3B in the traveling direction of the image light transmitted through the polarization separation member 101B.

[0231] 14(2), the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 is longer than the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 in FIG. 14(1). This is because in the optical system of FIG. 14(2), an optical path going back and forth between the polarization separation member 101B and the specular reflector 4, which does not exist in the optical system of FIG. 14(1), is added to the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2.

[0232] As a result, in the optical system of Figure 14(2), the distance from the position where the central light beam of the image light passes through the polarization separation member 101B to the position where the image light forms the space-floating image 3B (L2 in the example of Figure 14(2)) is significantly longer than the distance from the position where the central light beam of the image light is reflected by the polarization separation member 101B to the position where the image light forms the space-floating image 3A (L1 in the example of Figure 14(1)) in the optical system of Figure 14(1).

[0233] In addition, in the polarization design of the optical system in Fig. 14(2), the characteristics of P polarization and S polarization may be interchanged. Specifically, a predetermined polarization of the image light emitted from the display device 1 may be S polarization, and the characteristics of P polarization and S polarization may be interchanged in the reflection characteristics of the polarization separation member 101B. In this case, the illustrated P polarization and S polarization are both reversed, but the optical design such as the optical path can be realized in exactly the same way.

[0234] According to the optical system of Fig. 14(1) and Fig. 14(2) in the embodiment 2 of the present invention described above, a more compact optical system can be realized. In particular, according to the optical system of Fig. 14(2), it is possible to increase the amount of the floating image projecting from the optical system, while being a more compact optical system.

[0235] In addition, when the optical system of Fig. 14(1) or Fig. 14(2) is incorporated into a space-floating image display device, it can be realized by replacing the optical system in the space-floating image display device described in the first embodiment with the optical system of Fig. 14(1) or Fig. 14(2). Specifically, the optical system of Fig. 14(1) may be replaced with the optical system of the space-floating image display device of Fig. 4E, Fig. 4F, Fig. 4G, Fig. 4H, Fig. 4I, Fig. 4J, Fig. 4K, Fig. 4L, or Fig. 4M. In this case, since the optical system becomes compact, it is possible to make the housing of the space-floating image display device of each figure smaller.

[0236] Specifically, the optical system of FIG. 14(2) may be replaced with the optical system of the space-floating image display device of FIG. 4E, FIG. 4F, FIG. 4G, FIG. 4K, or FIG. 4L. In this case, it is possible to increase the amount of the space-floating image projected from the optical system. Also, since the optical system becomes compact, it is possible to make the housing of the space-floating image display device of each figure smaller.

[0237] <Example 3> As the third embodiment of the present invention, a configuration example of a space-floating image display device will be described. The space-floating image display device of the third embodiment can be applied with the configuration of the first embodiment or the second embodiment as a basic configuration. The space-floating image display device of the third embodiment can be applied with the configuration of the optical system based on, for example, FIG. 2A, etc., and the configuration of the housing, in other words, the main body based on, for example, FIG. 4A. Moreover, the third embodiment is a system (sometimes referred to as a space-floating image display system, etc.) having a space-floating image display device and an external device connected thereto. The external device is, for example, a mobile terminal such as a smartphone, tablet, or wearable terminal (smart watch, etc.) owned by a user. The mobile terminal may be called a mobile information processing terminal device.

[0238] The basic functions realized in the space-floating image display system of the third embodiment are as follows. This space-floating image display system connects and cooperates with the space-floating image display and the user's mobile terminal through communication. This system has a function of displaying an image / image owned by the user, for example, an image (sometimes referred to as a target image, etc.) that is in the mobile terminal and that the user is displaying on the screen of the mobile terminal, as a space-floating image 3 on the space-floating image display device. To this end, the user performs an operation / motion to bring the mobile terminal into contact with the space-floating image 3 (a corresponding predetermined imaging position, display range) of the space-floating image display device. This operation / motion is a predetermined operation / motion, and sometimes referred to as an inserting operation / contacting operation, etc. The space-floating image display device judges and detects the contacting operation using a sensor, etc. In this embodiment, for example, while the mobile terminal transmits a display request based on the user's input operation, the user performs a contacting operation to insert / contact the mobile terminal with respect to the plane position of the space-floating image 3.

[0239] Based on the contact operation, the space-floating image display device or the mobile terminal generates a display request / display instruction to display the target image on the mobile terminal side as the space-floating image 3. In other words, when the space-floating image display device / mobile terminal determines that a predetermined contact operation has been performed that satisfies a predetermined condition regarding the contact between the mobile terminal and the space-floating image 3, it generates such a display request / display instruction. The image control unit 1160 of the space-floating image display device, particularly in FIG. 3, receives the display request and receives / acquires the data of the target image from the mobile terminal side by communication. Then, the space-floating image display device executes image processing of the display device 1 so as to display the target image as the space-floating image 3.

[0240] This function may be as follows. The user's mobile terminal transmits the display request together with the above contact operation, in other words, immediately before, during, or after the contact. The space-floating image display device receives and permits / enables the display request from the mobile terminal based on the above contact operation, and executes the process of displaying the target image as a space-floating image 3 in the same manner as above. The display request may be set together with the transmission of the data of the target image. This function differs in whether the display request is generated by the space-floating image display device or the mobile terminal, but in either case, the function is the same in that the target image is automatically displayed as a space-floating image 3 in response to a specific contact operation.

[0241] This function may be as follows. Initially, the mobile terminal does not transmit a display request, and the user touches the mobile terminal to the floating-in-space image 3. When the floating-in-space image display device determines and detects a specific touch operation, it transmits information such as a confirmation to the mobile terminal via communication that it accepts the display request, in other words, that the image can be displayed using this function. When the mobile terminal receives the information such as the confirmation, it transmits data of the target image to the floating-in-space image display device. The data of the target image may be accompanied by a display request. When the floating-in-space image display device receives the data of the target image, it displays the target image as the floating-in-space image 3.

[0242] [Floating Space Image Display System] FIG. 15 shows the configuration of a system 3000, which is a space-floating image display system of the third embodiment. This system 3000 is a system in which a space-floating image display device 1000 and a mobile terminal 2000 of a user 230 are connected via a communication network such as a LAN 3020 and / or the Internet 3010. FIG. 15 shows a schematic diagram of a housing 1190 of the space-floating image display device 1000, in other words, a main body, a space-floating image 3, and a part of a communication unit 1132 (FIG. 3). The housing 1190 is equipped with each of the components as shown in FIG. 3 above. Note that the control unit 1110 and the communication unit 1132 of FIG. 3 may be installed inside the housing 1190 or outside the housing 1190. The camera of the imaging unit 1180, the air operation detection sensor 1351, a microphone, a speaker, and the like may be installed outside the housing 1190.

[0243] In this system 3000, communication is performed between the mobile terminal 2000 and the space-floating image display device 1000 through an arbitrary communication interface. The mobile terminal 2000 and the space-floating image display device 1000 may be directly connected for communication, for example, by short-distance wireless communication, or may be connected between the mobile terminal 2000 and the space-floating image display device 1000 via the Internet 3010 or the like. In this system 3000, image data and the like are exchanged between the mobile terminal 2000 and the space-floating image display device 1000, so a certain level of communication speed is required. Therefore, in the example of FIG. 15, the mobile terminal 2000 and the space-floating image display device 1000 are connected through a LAN 3020. That is, the mobile terminal 2000 and the space-floating image display device 1000 are connected to the same network through a router or the like, and are set so that they can be identified from each other using IP addresses or the like. This is not limited to this, and communication by mirroring or the like is also possible.

[0244] The mobile terminal 2000 has an application 2010 and an image 2020. The application 2010 is, for example, an arbitrary application program that displays an image 2020 or the like on the screen of the display of the mobile terminal 2000. The application 2010 may be a general OS or app, or may be a dedicated application for linking and communicating with the space-floating image display device 1000 of the system 3000 of the third embodiment. The data of the application 2010 may be distributed from the space-floating image display device 1000, or may be distributed from a server or the like on the Internet 3010. The image 2020 is image data that is a target image. The image 2020 may be held in a server or the like on the Internet 3010, or may be distributed from the space-floating image display device 1000. The image 2020 may be a still image or a video. The image 2020 may be data in a format such as a program.

[0245] The application 2010 of the mobile terminal 2000 may create a display request 2030 for displaying the target image (image 2020) as the space-floating image 3, and transmit the request to the space-floating image display device 1000.

[0246] 16 shows an overview of the functions of the space floating image display system of the embodiment 3. (1) First, the user 230 has a target image 1601 and the like in his / her mobile terminal 2000. The target image 1601 is an image to be displayed as the space floating image 3.

[0247] (2) The user 230 performs a contact operation 1600 of inserting the mobile terminal 2000 into and contacting the display range 3R of the space-floating image 3 of the space-floating image display device 1000. At this time, the mobile terminal 2000 may transmit a display request 2030. The display request 2030 is a request or instruction for displaying the target image 1601 as the space-floating image 3.

[0248] (3) When the space-floating image display device 1000 detects the contact operation 1600, it displays the target image 1601 of the mobile terminal 2000 as an image 1602, which is a space-floating image 3. At this time, the space-floating image display device 1000 may receive a display request 2030 and obtain the target image 1601 of the mobile terminal 2000 based on the display request 2030. The space-floating image display device 1000 may transmit some kind of response to the display request 2030 to the mobile terminal 2000.

[0249] 17 shows an example of the configuration of a mobile terminal 2000. The mobile terminal 2000 includes a housing 2090, a control unit 20011, a display panel 20012, an external power input IF (interface) 20013, a power source 20014, a secondary battery 20015, a storage unit 20016, a video control unit 20017, a position sensor 20018, an operation input unit 20019, a communication unit (including an antenna) 20020, an audio output unit (including a speaker) 20021, an audio input unit (including a microphone) 20022, a video signal input unit 20023, an audio signal input unit 20024, an imaging unit 20025, a memory 20026, a non-volatile memory 20027, and the like. These components are connected by an architecture such as a bus.

[0250] The control unit 20011 has a processor or the like that controls the entire mobile terminal 2000 and each unit. The control unit 20011 loads data such as a program stored in the non-volatile memory 20027 or the storage unit 20016 into the memory 20026 and executes processing according to the program. This realizes various functions. The control unit 20011 may cooperate with the program stored in the memory 20026 to perform arithmetic processing based on information acquired from each unit in the mobile terminal 2000. The application 2010 in FIG. 15 is executed by, for example, the control unit 20011 or the video control unit 20017.

[0251] The memory 20026 stores image data to be displayed on the display panel 20012, control data for the mobile terminal 2000, etc. The control unit 20011 may read out various software programs from the storage unit 20016, etc., and develop and store them in the memory 20026. The non-volatile memory 20027 and the storage unit 20016 hold various data and information used by the mobile terminal 2000. The data and information stored in the non-volatile memory 20027 and the storage unit 20016 include, for example, data for various operations to be displayed on the display panel 20012, display icons, data and layout information for objects to be operated by the user 230, etc.

[0252] The display panel 20012 is a display unit, for example a touch panel, which is a display means and an input means for accepting touch operation input. Images and the like are displayed on the screen of the display panel 20012. The display panel 20012 is provided with a touch sensor and accepts touch operation input by the finger of the user 230 or the like. The display panel 20012 can be a liquid crystal panel, an organic EL panel, or the like.

[0253] The communication unit 20020 is a device in which various communication interfaces are implemented. The communication unit 20020 is implemented with, for example, a mobile communication interface such as 4G or 5G, a wireless LAN communication interface such as Wi-Fi (registered trademark), or a short-range communication interface such as Bluetooth (registered trademark) or NFC, as a communication interface / communication method. The communication unit 20020 of the mobile terminal 2000 can communicate with the communication unit 1132 (FIG. 3) of the space-floating image display device 1000 using these communication methods. Similarly, from the perspective of the space-floating image display device 1000, the communication unit 1132 is implemented with these communication interfaces / communication methods. The communication unit 20020 can communicate with a communication device connected to the Internet 3010, such as a wireless base station, using any of the communication methods. This allows the mobile terminal 2000 to communicate with a server connected to the Internet 3010.

[0254] The power supply 20014 converts an AC current input from the outside via an external power supply input IF (interface) 20013 into a DC current, and supplies the required DC current to each unit of the mobile terminal 2000. The secondary battery 20015 stores the power supplied from the power supply 20014. When power is not supplied from the outside via the external power supply input IF 20013, the secondary battery 20015 supplies power to each unit requiring power.

[0255] The video signal input unit 20023 connects to an external video output device and inputs video data. The video signal input unit 20023 can be applied with various digital video input interfaces. For example, a video input interface conforming to the HDMI (registered trademark) (High-Definition Multimedia Interface) standard, a video input interface conforming to the DVI (Digital Visual Interface) standard, or a video input interface conforming to the DisplayPort standard can be applied. Alternatively, an analog video input interface such as analog RGB or composite video may be provided. The video signal input unit 20023 may be various USB interfaces or the like.

[0256] The audio signal input unit 20024 is connected to an external audio output device and inputs audio data. The audio signal input unit 20024 can be an HDMI-standard audio input interface, an optical digital terminal interface, a coaxial digital terminal interface, or the like. The audio signal input unit 20024 may be any of various USB interfaces. In the case of an HDMI-standard interface, the video signal input unit 20023 and the audio signal input unit 20024 may be configured as an interface in which a terminal and a cable are integrated.

[0257] The audio output unit 20021 can output audio based on audio data input to the audio signal input unit 20024 or audio data stored in the storage unit 20016. The audio output unit 20021 may be configured with a speaker or may have an earphone terminal. The audio output unit 20021 may also output built-in operation sounds and error warning sounds. The audio output unit 20021 may be configured to output digital signals to an external device, such as an Audio Return Channel function defined in the HDMI standard.

[0258] The audio input unit 20022 including a microphone collects sounds around the mobile terminal 2000, converts them into signals, and generates audio signals. The audio input unit 20022 may generate audio signals by collecting the voice of a person, such as the voice of the user 230, with the microphone, and the control unit 20011 or the like may perform voice recognition processing on the audio signal to obtain text information.

[0259] The imaging unit 20025 is, for example, a camera having an image sensor. The mobile terminal 2000 may have a camera (in-camera) on the front side of the housing 2090, which is the side of the display panel 20012, or a camera (out-camera) on the opposite back side. In this embodiment, the imaging unit 20025 has both an in-camera and an out-camera.

[0260] The storage unit 20016 is a storage device that records various data such as video data, image data, and audio data, and various information such as programs. The storage unit 20016 may be configured, for example, with a magnetic recording medium recording device such as a hard disk drive (HDD) or a semiconductor element memory such as a solid state drive (SSD). For example, various data such as video data, image data, and audio data, and various information such as programs may be recorded in the storage unit 20016 beforehand at the time of product shipment. The storage unit 20016 may also record various data and information acquired from an external server or the like via the communication unit 20020. The image data and the like recorded in the storage unit 20016 are output and displayed on the display panel 20012 through processing by the video control unit 20017. The mobile terminal 2000 may output and transmit various data and information recorded in the storage unit 20016 to an external server or the like via the communication unit 20020.

[0261] The video control unit 20017 performs various controls related to the video signal input to the display panel 20012. The video control unit 20017 may be called a video processing circuit, a video processing unit, an image processing unit, etc. The video control unit 20017 may be configured with hardware such as an ASIC, an FPGA, or a video processor. The video control unit 20017 performs video switching control, for example, to determine which video signal is input to the display panel 20011 and displayed among the video signals to be stored in the memory 20026, etc., and the video signals (video data) input to the video signal input unit 20023, etc. The video control unit 20017 may also perform control to perform image processing on the video signals input from the video signal input unit 20023 and the video signals to be stored in the memory 20026, etc. Examples of image processing include scaling processing to enlarge, reduce, deform, etc. an image, brightness adjustment processing to change the brightness, contrast adjustment processing to change the contrast curve of the image, and Retinex processing to break down an image into light components and change the weighting of each component. The video control unit 20017 may include a video memory or the like that holds video data to be input to the display panel 20012 .

[0262] The attitude sensor 20018 is a sensor configured by combining a gyro sensor, a gravity sensor, an acceleration sensor, a geomagnetic sensor, etc., and can detect the attitude of the mobile terminal 2000. The attitude can be expressed, for example, by the directions and angles of three orthogonal axes (X, Y, Z) in space. Based on the attitude detection result of the attitude sensor 20018, the control unit 20011 may control the operation of each unit. In addition, the mobile terminal 2000 may include a GPS receiver, a proximity sensor, an illuminance sensor, a distance sensor, etc.

[0263] The operation input unit 20019 is a device for the user 230 to input operations, and includes, for example, a power button and a volume button.

[0264] [Video display method] Fig. 18 (Fig. 18A, etc.) is an explanatory diagram of an image display method by cooperation between the space-floating image display device 1000 and the mobile terminal 2000 in the system 3000 of the third embodiment. Fig. 18A, etc. show a state in which a user 230 makes a display request 2030 from the mobile terminal 2000 to the space-floating image display device 1000 by touching the mobile terminal 2000 to the display range 3R of the space-floating image 3 of the space-floating image display device 1000. Fig. 18A is a perspective view, Fig. 18B is a top view (XY plane view) seen from above, Fig. 18C is a side view (YZ plane view) seen from the side, and Fig. 18D is a front view (XZ plane view) seen from the front of the device.

[0265] In FIG. 18A, the upper surface of the housing 1190 has an opening 1200, and the opening 1200 is provided with a transparent member 100 (for example, a glass plate) and a polarization separation member 101, for example, as in FIG. 2A. The display range 3R represents a predetermined position or plane in the space where the real image, the floating-in-space image 3, is formed. In FIG. 18A, the spatial coordinate system is represented by (X, Y, Z). The X-axis and X-directions are the left-right direction (first horizontal direction) as seen from the user 230 (a viewpoint not shown at the origin of the direction of the arrow A). The Y-axis and Y-directions are the front-back direction (depth direction, second horizontal direction) as seen from the user 230. The Z-axis and Z-directions are the up-down direction (vertical direction) as seen from the user 230. The coordinate system in the floating-in-space image 3 is represented by (x, y, z). The x-direction is the horizontal direction of the screen, the y-direction is the vertical direction of the screen, and the z-direction is the depth direction.

[0266] A direction 1801 shows an example of a direction (insertion direction) in which the user 230 inserts the mobile terminal 2000 into the display range 3R of the floating-in-space image 3 when performing a contact operation, and in this example, it is the Y direction. In Fig. 18A, an example of a place where the mobile terminal 2000 and the display range 3R come into contact is shown as a contact point 1802. In this example, the contact point 1802 is near the bottom right of the display range 3R.

[0267] In the embodiment of FIG. 18, the display request 2030 is made from the mobile terminal 2000 to the space-floating image display device 1000 by contacting the mobile terminal 2000 with the display range 3R. In other words, the display request 2030 is sent from the mobile terminal 2000 to the space-floating image display device 1000 immediately before, during, or immediately after the contact. For example, the space-floating image display device 1000 judges and detects this contact, and upon detecting this contact, receives the display request 2030 transmitted from the mobile terminal 2000. That is, the space-floating image display device 1000 allows the target image to be displayed on the space-floating image 3 based on the display request 2030. The space-floating image display device 1000 receives image data of the target image associated with the display request 2030 from the mobile terminal 2000 by transfer. The space-floating image display device 1000 controls the display device 1 based on the acquired image data to display the target image as the space-floating image 3. In other words, the space-floating image display device 1000 converts the target image designated by the user 230 into a space-floating image and displays it as the space-floating image 3.

[0268] The display request 2030 transmitted from the mobile terminal 2000 can be transmitted by the various communication methods described above, and the details are not limited. In this example, the display request 2030 is transmitted by wireless communication in the LAN 3020, but the present invention is not limited to this, and a wired connection may be used, or the mobile terminal 2000 and the space floating image display device 1000 may be connected by infrared communication or the like.

[0269] The determination and detection of contact between the mobile terminal 2000 and the display range 3R will be described later, but examples of the determination and detection include an aerial operation detection sensor 1351 (FIG. 3) and an imaging unit 1180 (camera). In addition, the determination and detection of the contact may be performed using various sensors attached inside and outside the housing 1190, or various sensors included in the mobile terminal 2000, such as a gyro sensor, or a combination of these.

[0270] As described above, in this system 3000, when the mobile terminal 2000 judges and detects a touch operation on the display range 3R, the space-floating image display device 1000 recognizes the display request 2030 from the mobile terminal 2000, and the space-floating image display device 1000 starts displaying the target image of the space-floating image. As described above, a feature of the third embodiment is that the target image, which is a space-floating image, can be displayed simply by touching and placing the mobile terminal 2000 on the display range 3R of the space-floating image 3.

[0271] [Touch operation] FIG. 19 (FIG. 19A, etc.) shows the attitude change direction when the mobile terminal 2000 is brought into contact with the display range 3R of the floating-in-space image 3. There are various attitude change directions 1901 of the mobile terminal 2000 as shown in the figure. FIG. 19A shows the rotation angles of three axes, pitch angle α, roll angle β, and yaw angle γ, in a perspective view. In this embodiment, as a rule, the attitude of the mobile terminal 2000 can be arbitrary at the time of the contact operation, that is, the state of these angles (α, β, γ) can be any state. This simplifies the contact operation by the user 230, and is highly convenient.

[0272] FIG. 19B is an XY plane view, and shows an example of a case where the attitude of the mobile terminal 2000 is rotated in a horizontal plane (XY plane) during a contact operation. For the sake of explanation, a representative position and orientation of the mobile terminal 2000 is the front of the housing, particularly the central position P1 of the display panel 20012, and the orientation R1 (same as the axis of the roll angle β) along the long side of the front, as shown in FIG. 19A. In this example, the orientation R1 of the mobile terminal 2000 at the time of contact and the corresponding insertion direction 1801 are rotated around the Z axis (yaw angle γ) and are inclined diagonally to the left with respect to the Y axis. The attitude change direction 1901 is the rotation direction around the Z axis (yaw angle γ). The vicinity of the upper side of the housing of the mobile terminal 2000 is in contact with the display range 3R (contact point 1802). For example, the user 230 inserts the mobile terminal 2000 held in the right hand into the vicinity of the center of the display range 3R.

[0273] 19C is a YZ plane view, and shows an example of a case where the attitude of the mobile terminal 2000 is rotated in the YZ plane during a contact operation. In this example, the orientation R1 of the mobile terminal 2000 at the time of contact and the corresponding insertion direction 1801 are rotated around the X axis (pitch angle α) and are inclined diagonally downward with respect to the Y axis. The attitude change direction 1901 is the rotation direction around the X axis (pitch angle α). In this case, the insertion direction 1801 is perpendicular to the display range 3R. The vicinity of the upper edge of the housing of the mobile terminal 2000 is in contact with the display range 3R (contact point 1802).

[0274] 19D is an XZ plane view, and shows an example of a case where the attitude of the mobile terminal 2000 is rotated in the XZ plane during a touch operation. In this example, the orientation R1 of the mobile terminal 2000 during contact and the corresponding insertion direction 1801 are rotated around the Y axis (roll angle β) and the flat plate of the housing is oriented vertically. The attitude change direction 1901 is the rotation direction around the Y axis (roll angle β).

[0275] FIG. 19E is a YZ plane cross-sectional view, and shows another example of the attitude of the mobile terminal 2000 during a touch operation. This example shows a case where the screen of the display panel 20012 on the front side of the mobile terminal 2000 faces downward during a touch operation. FIG. 19E shows an example in which the configuration of FIG. 4A is applied as the configuration of the space-floating image display device 1000. Image light is emitted obliquely upward from the opening 1200 of the transparent member 100 on the upper surface (XY plane) of the housing 1190, forming the space-floating image 3. The plane (xy plane) of the display range 3R of the space-floating image 3 is arranged at an angle of about 45 degrees from the XY plane. In FIG. 19E, the mid-air operation detection sensor 1351 and the like are omitted (described later).

[0276] When performing a contact operation, user 230 inserts portable terminal 2000 with screen 1902 facing downward (direction 1903) into display range 3R in insertion direction 1801 toward the back (positive) in the Y direction. FIG. 19E shows a state (portable terminal 2000b) in which portable terminal 2000 has contacted display range 3R and passed to the back side. In this state, screen 1902 of portable terminal 2000b faces downward (direction 1903). Such a contact operation may be performed.

[0277] Fig. 19F is a YZ plane cross-sectional view, showing another example of the attitude of the mobile terminal 2000 during a contact operation. This example shows a case where the screen 1902 of the display panel 20012 on the front side of the mobile terminal 2000 is oriented along the plane (xy plane) of the display range 3R of the floating-in-space image 3 during a contact operation. The user 230 operates the screen 1902 of the mobile terminal 2000 so as to face and overlap the display range 3R. In Fig. 19F, the screen 1902 of the mobile terminal 2000 is maintained in contact with the display range 3R.

[0278] The predetermined touch operation may be determined based on the time (contact time) that the mobile terminal 2000 touches the display range 3R. That is, the space floating image display device 1000 counts the contact time, and when the contact time reaches or exceeds a certain time as a determination condition, it determines that the predetermined touch operation has been performed.

[0279] 20 (FIG. 20A, etc.) are explanatory diagrams showing that when a contact operation is performed to insert the portable terminal 2000 into the display range 3R, the position of the portable terminal 2000 may basically be any position at each point including before, during, and after the insertion. The insertion of the portable terminal 2000 into the display range 3R only needs to be in contact with the portable terminal 2000 and the display range 3R, in other words, only needs to be capable of contact determination, and the portable terminal 2000 may be in any position, insertion direction, or attitude angle. Furthermore, the contact between the display range 3R and the portable terminal 2000 may basically be performed at any point in the display range 3R.

[0280] In this way, by broadening the range of permitted touch operations and actions (corresponding judgment conditions), there is an advantage that user operation becomes easier. Conversely, it is also possible to narrow the range of permitted touch operations and actions (judgment conditions). In other words, the position, posture, insertion direction, contact time, contact area of ​​the display range 3R, etc., of the mobile terminal 2000 permitted during a touch operation (before insertion, during insertion, after insertion, etc.) may be specified to be limited.

[0281] Fig. 20A shows examples of the position and insertion direction of the portable terminal 2000 before insertion on the YZ plane. In Fig. 20A, the illustration of the portable terminal 2000 is omitted, and the position of the portable terminal 2000 is shown by a dot. Black and white circles (p1, etc.) show examples of the position of the portable terminal 2000 before insertion. As shown in the figure, various positions and insertion directions are possible as long as the portable terminal 2000 contacts the display range 3R.

[0282] 20B shows examples of the position and insertion direction of mobile terminal 2000 before insertion on the XY plane. An operation can be performed in any insertion direction so as to touch display range 3R from any position before insertion (such as p11). In addition, in the case of position p14, before insertion, mobile terminal 2000 is on the right side of display range 3R and is further back in the Y direction than display range 3R, and the insertion direction is from the back (+Y) to the front (-Y) or left and right with respect to display range 3R. Even in such a case, if mobile terminal 2000 touches display range 3R, it is permitted as a touch operation.

[0283] 20C is a YZ plan view showing an example of a state when the mobile terminal 2000 touches the display range 3R. As long as the touch can be detected and determined, the touch of the mobile terminal 2000 with the display range 3R may be a touch of any part of the mobile terminal 2000. In addition, after the mobile terminal 2000 touches the display range 3R once (after a touch operation is detected), the mobile terminal 2000 may move out of the display range 3R.

[0284] State 2000c1 shows an example of the state of the mobile terminal 2000 at the time of contact, and the contact location (contact position) c1 at that time is near the center on the display range 3R side, and is, for example, a part near the upper side of the housing on the mobile terminal 2000 side. State 2000c2 shows another example of the state of the mobile terminal 2000 at the time of contact, and the contact location c2 at that time is at the upper part of the display range 3R, and is, for example, a part near the lower side of the housing. State 2000c3 shows another example of the state of the mobile terminal 2000 at the time of contact, and the contact location c3 at that time is at the lower part on the display range 3R side, and after contact once, the mobile terminal 2000 passes to the rear side and does not touch the display range 3R. State 2000c4 is a state in which the mobile terminal 2000 is returned to the front side in the Y direction from the contact state 2000c1.

[0285] [Variation: Judgment Condition] In the third embodiment, the posture and position of the mobile terminal 2000 when inserting and touching the display range 3R are arbitrary, but the present invention is not limited to this. In a modified example, the posture and position of the mobile terminal 2000 when inserting and touching the display range 3R may be limited to a specific state. That is, the space floating image display device 1000 may determine and detect a specific posture or other state when the mobile terminal 2000 is touched, and allow the display request 2030 only when the state is detected. As a modified example, when narrowly limiting the allowable predetermined touch operation, for example, the following determination conditions can be mentioned.

[0286] (1) Only the insertion direction from the front in the Y direction, that is, from the side where the user 230 views the floating image 3, toward the back, is permitted with respect to the plane of the display range 3R of the floating image 3. In other words, contact with the display range 3R from the back or side is not permitted. Also, only the insertion direction within a specified angle range is permitted with respect to the plane of the display range 3R of the floating image 3.

[0287] (2) When the mobile terminal 2000 touches the plane of the display range 3R, the contact location, i.e., the position of the mobile terminal 2000, the area within the display range 3R is limited to a part, such as a central part, a part at the bottom right, or a part at the top left within the display range 3R.

[0288] (3) The posture of the mobile terminal 2000 when the mobile terminal 2000 contacts the plane of the display range 3R is limited to some postures. For example, the angle with respect to the plane of the display range 3R is within a predetermined angle range. For example, as shown in FIG. 19C, only the case where the flat housing of the mobile terminal 2000 is inserted approximately perpendicular to the plane of the display range 3R may be allowed. Also, for example, as shown in FIG. 19F, only the case where the flat housing of the mobile terminal 2000 is contacted approximately parallel to the plane of the display range 3R may be allowed.

[0289] (4) The attitude and holding time of the mobile terminal 2000 in a state after the mobile terminal 2000 comes into contact with the plane of the display range 3R and then passes to the rear side are limited. For example, as in the example of FIG. 19E, the attitude angle with respect to the plane of the display range 3R or the opening 1200 is within a predetermined angle range.

[0290] [Guide display] 21A shows an example in which, when performing a touch operation to insert the mobile terminal 2000 into the display range 3R of the space-floating image 3, the space-floating image display system displays guide information for the user 230 regarding the insertion / touch operation. In the third embodiment, the space-floating image display device 1000 displays the guide information, in other words, information for guiding and instructing the location and direction to insert the mobile terminal 2000, and user interface information, on the space-floating image 3. This allows the user 230 to easily perform the touch operation to insert the mobile terminal 2000 into the display range 3R according to the guide information.

[0291] Examples of guidance displays in this guide information include the following. (a) Displaying the entire display range 3R (the area that accepts touch operations; a partial area is also acceptable) in a predetermined color, etc. Alternatively, displaying a frame representing the entire display range 3R in color, etc. (b) Explain the insertion location and direction with arrows or other markings. (c) Explain the insertion location and direction using natural language, video, images, animation, etc.

[0292] In the example of FIG. 21A, a display example of guide information is shown at the bottom in an xy plane view when the floating image 3 is viewed from the front. The guide information of FIG. 21A illustrates several guidance display examples together, and at least one of these may be applied. Guidance display example 2101 is an example in which a rectangular frame representing the entire display range 3R is displayed in color. Guidance display example 2102 is an example in which an area that accepts insertion and contact is shown to the user by displaying an arrow image (which may be another predetermined symbol or figure). The arrow image in this example is an image of multiple arrows pointing inward from the four sides of the display range 3R. Guidance display example 2103 is an example in which a character image in natural language, for example, "Insert here", is displayed to inform the user of insertion and contact in the area of ​​the display range 3R. Guidance display example 2104 is an example in which an image or image, for example, animation, is used to inform the user of insertion and contact in the area of ​​the display range 3R.

[0293] As a modified example, as described later, guide information may be similarly displayed on the screen of the mobile terminal 2000.

[0294] As a modified example, a guide image for conveying the contact status may be displayed on the floating-in-space image 3 when performing an insert / touch operation on the mobile terminal 2000. In the example of Fig. 21B, only a part of the display range 3R of the floating-in-space image 3, 2105, is defined as a predetermined area (insertion area / contact area) that accepts the insert / touch operation.

[0295] The space floating image display device 1000 detects the contact state of the mobile terminal 2000 with respect to the display range 3R using a sensor, and displays a guide image 2106 representing the contact state according to the detected contact location. This guide image 2106 is an image representing that the mobile terminal 2000 is in contact with the display range 3R at a spatial position corresponding to the guide image 2106. In this example, the guide image 2106 is an effect image like a ripple spreading from the contact location. By looking at the guide image 2106, the user 230 can more easily recognize the contact state of the mobile terminal 2000 with the display range 3R. Then, the user 230 may move the mobile terminal 2000 to the insertion area, which is a part of the area 2105, based on the guide image 2106. The guide information / feedback information to the user 230 is not limited to the image display as described above, and may be used in combination with audio output or the like.

[0296] [Display restriction function] In addition, the present system 3000 may have the following function (hereinafter, referred to as a display restriction function). When the mobile terminal 2000 is inserted into the display range 3R during the contact operation, the present system 3000 may hide the space-floating image 3 in a part of the display range 3R near the row in which the mobile terminal 2000 is inserted, and then perform another display in the remaining area of ​​the display range 3R. When the user 230 looks at the display range 3R due to the operation of inserting the mobile terminal 2000, there may be some parts where the space-floating image 3, which is the display image, is not visible due to light blocking or shading by the mobile terminal 2000. This function is a function that controls the display of the space-floating image 3 in such a part (partial area) that is not visible so that it is temporarily hidden. In this function, the image control unit 1160 of the space-floating image display device 1000, particularly in FIG. 3, may realize the above-mentioned non-display by controlling the light source device 13 and the image display element (liquid crystal display panel 11) of the display device 1.

[0297] In addition, when a portion of the display range 3R is made invisible due to the above-mentioned contact operation, the space-floating image display device 1000 may reduce the display size or change the aspect ratio of the displayed image so that the remaining display area matches the space-floating image 3, which is the displayed image.

[0298] FIG. 22 (FIG. 22A, etc.) shows an example of this display restriction function. FIG. 22A is an XY plane view, and shows a case where the user 230 inserts and touches the mobile terminal 2000 in the Y direction with respect to a part of the display range 3R, for example, similarly to FIG. 18A to FIG. 18D. An area 2201 shows the position and area of ​​the mobile terminal 2000 in a state of contact after insertion into the display range 3R. FIG. 22B shows an XZ plane view corresponding to the state of FIG. 22A. An area 2202 is an example of an unrecognizable area caused by the blocking of the image light. Correspondingly, FIG. 22C shows a state of blocking the image light in a YZ plane view. A position 2203 shows the position and area of ​​the mobile terminal 2000 in a state of contact after insertion into the display range 3R.

[0299] Image light component 2204 indicates a component of the luminous flux of image light emitted to the outside from opening 1200 that is not blocked by mobile terminal 2000 (position 2203) and enters the viewpoint (eye) of user 230, thereby forming space floating image 3 in display range 3R. Image light component 2205 indicates a component of the luminous flux of image light emitted to the outside from opening 1200 that is blocked by mobile terminal 2000 (position 2203) and does not enter the viewpoint (eye) of user 230, thereby forming invisible area 2202 in display range 3R.

[0300] FIG. 23 (FIG. 23A, etc.) shows a display example of a guide image in the space floating image 3, as well as an operation example of the display restriction function.

[0301] 23A shows an example in which a guide image 2302 indicating that a display request 2030 is occurring is displayed when a contact operation is detected in the front view (xy plane) of the display range 3R of the floating-in-space image 3. In this example, a frame display 2301 showing the whole is present in the display range 3R, and a guide image 2302 indicating that a display request 2030 is occurring is displayed within the frame display 2301. The guide image 2302 is, for example, a text image indicating that a display request 2030 is occurring, such as "Requesting..." or an arrow icon. In this state, an invisible area 2202 is generated due to occlusion by the mobile terminal 2000.

[0302] 23B shows an example in which the space floating image display device 1000 hides a part of the display range 3R and controls the display of the remaining area based on the detection of the occurrence of the invisible area 2202 as shown in FIG. 23A. The occurrence of the invisible area 2202 can be detected by the mid-air operation detection sensor 1351 or the imaging unit 1180.

[0303] For example, the image control unit 1160 of the space floating image display device 1000 in FIG. 3 determines and detects the invisible area 2202 together with the contact operation, and determines the non-display area including the invisible area 2202 in the display range 3R. In this example, based on the detection by the midair operation detection sensor 1351 (described later), the touch location 2303 in the X direction and the x direction, in other words, the width and the row, are known in the display range 3R. Therefore, the space floating image display device 1000 determines the display row corresponding to the touch location 2303 in the display range 3R to be the non-display area 2304 shown by the diagonal hatching pattern. The non-display area 2304 includes the invisible area 2202. The area of ​​the display range 3R other than the non-display area 2304 becomes the display area 2305. In other words, the space floating image display device 1000 configures the display area 2305 by reducing the size of the display range 3R in the X-direction and the x-direction by the width of the non-display area 2304.

[0304] The space-floating image display device 1000 controls the display of the display device 1 so that the space-floating image 3 is not displayed in the non-display area 2304, and is displayed only in the display area 2305. In this example, in the display area 2305 of the display range 3R, the original frame image 2301 and guide image 2302 as shown in FIG. 23A are changed to a frame image 2306 and a guide image 2307 by being adjusted, for example reduced, so as to fit within the width of the display area 2305 while maintaining the aspect ratio.

[0305] The above function allows the user 230 to easily view the guide image 2307 and the like in the space floating image 3. In the above example, a larger area is set as the non-display area 2304 compared to the invisible area 2202, but the present invention is not limited to this. The non-display area may be the same area as the invisible area, or may be a different area that overlaps at least partially with the invisible area. The guide image can be changed or adjusted in various ways, such as changing the aspect ratio, changing the color or brightness, and shifting the display position.

[0306] 24A shows an example of displaying a guide image of the floating-in-space image 3 as feedback such as a notification to the user 230 when a predetermined touch operation is detected, in other words, the touch operation is determined to be successful, and a display request 2030 is received. In this example, a guide image 2401 indicating success is displayed over the entire display range 3R. The guide image 2401 is composed of characters, icons, etc. indicating success. When the operation is successful in this way, the process moves to a phase in which the target image is displayed as the floating-in-space image 3.

[0307] 24B shows an example of displaying a guide image of the floating-in-space image 3 as feedback such as a notification to the user 230 when a predetermined touch operation cannot be detected, in other words, when the touch operation is determined to have failed and the display request 2030 is not received. In this example, a guide image 2402 indicating failure is displayed over the entire display range 3R. The guide image 2402 is composed of characters, icons, etc. indicating failure or retry.

[0308] If the user 230 does not properly insert or touch the mobile terminal 2000 into the display range 3R, or if the user 230 properly inserts or touches the mobile terminal 2000 into the display range 3R but the system 3000 cannot properly detect the mobile terminal 2000 due to a sensor malfunction or the like, the space-floating image display device 1000 judges the operation as a failure or an error, as described above, and outputs a guide image 2402 as feedback to the user 230. In this case, the space-floating image display device 1000 may prompt the user 230 to retry the touch operation. The user 230 starts over from the touch operation.

[0309] 24C shows an example in which, when a guide image indicating that the display request 2030 from the mobile terminal 2000 to the space-floating image display device 1000 has been properly made (successful) together with a touch operation of the mobile terminal 2000 to the display range 3R is displayed as the space-floating image 3, the display area is adjusted according to the invisible area 2202. The non-display area 2404 is an area of ​​the display range 3R that includes the invisible area 2202, and the display area 2405 is an area other than the non-display area 2404.

[0310] (A) on the lower left side of Fig. 24C is an example of a successful case, and display area 2405 displays guide video 2401b whose size, etc. have been adjusted based on guide video 2401 as in Fig. 24A. Similarly, (B) on the lower right side of Fig. 24C is an example of an unsuccessful case, and display area 2405 displays guide video 2402b whose size, etc. have been adjusted based on guide video 2402 as in Fig. 24B.

[0311] The feedback to the user 230 may be not only visual, such as displaying a guide image in the floating image 3 or illuminating a lamp, but also auditory, such as audio output, or tactile, such as a device that generates stimuli such as vibration, air, or ultrasonic waves. Such devices, such as an air jet or ultrasonic generator, may be installed inside or outside the housing 1190. Also, each of the means may be used in combination at the same time.

[0312] FIG. 24D shows a case where the floating-in-space image display device 1000 generates a sound 2408 representing success or failure from the sound output unit 1140 in FIG. 3 such as a speaker to the user 230 when the touch operation and the display request 2030 are successful or unsuccessful on the YZ plane. (A) is a sound 2408 (a sound notification example) in the case of success, for example, "Connected." (B) is a sound 2408 (a sound notification example) in the case of failure, for example, "Could not connect" or "Please try again." The speaker may be installed inside or outside the housing 1190. The speaker may be a super-directional speaker or the like. The above example is an example of a sound output in a natural language, but is not limited thereto, and may be a predetermined short sound, for example, a beep, an alert sound, or music output.

[0313] [Air operation detection sensor] 25 (FIG. 25A, etc.) shows an implementation example of an aerial operation detection sensor 1351 and an imaging unit 1180 (camera) as sensors in the space-floating image display device 1000 of the embodiment 3. The aerial operation detection unit 1350, the control unit 1110, the image control unit 1160, etc. in FIG. 3 of the space-floating image display device 1000 of the embodiment 3 determine and detect the insertion and contact operation of the mobile terminal 2000 into the display range 3R of the space-floating image 3 using these sensors.

[0314] FIG. 25A is a YZ plan view showing an example of the arrangement of the aerial operation detection sensor 1351 and the imaging unit 1180 (camera) in the housing 1190. The configuration example of the optical system in FIG. 25A is the same as FIG. 4A and FIG. 18E. In the example of FIG. 25A, the aerial operation detection sensor 1351 is installed near the transparent member 100 on the upper surface of the housing 1190, on the front side close to the user 230 in the Y direction. The optical axis of this aerial operation detection sensor 1351 shown by the dashed line arrow is directed obliquely upward, that is, in the y direction, as shown in the figure, and the optical axis is set to overlap with the display range 3R of the floating in space image 3. Also, in the example of FIG. 25A, the imaging unit 1180 (camera) is installed in the housing 1190 with the optical axis shown by the dashed line arrow directed upward, that is, in the Z direction. The imaging range of the imaging section 1180 (camera) is set to include the display range 3R and the opening 1200, as indicated by the dashed lines.

[0315] FIG. 25B shows a configuration example of the aerial operation detection sensor 1351. FIG. 25B shows an xy plan view aligned with the display range 3R of the floating-in-space image 3. The aerial operation detection sensor 1351 has a plurality of optical elements 1351c arranged in the x direction (X direction in FIG. 25A). The optical element 1351c is a pair of a light-emitting element 1351a and a light-receiving element 1351b. The light-emitting element 1351a is composed of, for example, an infrared element. The emission surface of the light-emitting element 1351a of the aerial operation detection sensor 1351 coincides with the upper surface of the transparent member 100 of the housing 1190 in FIG. 25A. The light-emitting element 1351a emits light a1, for example, infrared light, in the y direction. If the light a1 is not blocked by an object, it passes through the display range 3R. If the light a1 is blocked by an object, it is reflected by the object and returns as reflected light a2. The reflected light a2 is received by the light receiving element 1351b.

[0316] For example, if there is a contact point 2501 by the finger of the user 230 within the xy plane of the display range 3R, light a1 is reflected at the contact point 2501 and returns as reflected light a2. The light receiving element 1351b at a certain x-direction position detects the reflected light a2. This allows the aerial operation detection unit 1350 to know that the contact point 2501 is at that x-direction position. Furthermore, the aerial operation detection unit 1350 can calculate the distance by the TOF method from the time it takes for the emitted light a1 to return as reflected light a2. For example, the distance 2502 to the contact point 2501 can be calculated. This allows the position coordinates of the contact point 2501 on the xy plane of the display range 3R to be known.

[0317] 25B shows a contact point 2503 when the mobile terminal 2000b is placed within the display range 3R by an insertion / touch operation. In this case, the aerial operation detection sensor 1351 and the aerial operation detection unit 1350 can detect the position coordinates of the contact point 2503 in the same manner. Note that, although there is nothing above the floating-in-space image 3 in the example of FIG. 25A, a part of the housing 1190 may be provided above the floating-in-space image 3. In that case, the light a1 of the aerial operation detection sensor 1351 is reflected by the part of the housing 1190 and returns as reflected light a2.

[0318] By using the aerial operation detection sensor 1351 and the aerial operation detection unit 1350, the floating-in-space image display device 1000 can detect the contact operation when the mobile terminal 2000b contacts the display range 3R as shown in Fig. 25A. Not limited to the above arrangement example, the aerial operation detection sensor 1351 may be arranged on the upper side of the xy plane of the display range 3R, or may be arranged at a position shifted forward or backward, that is, in the z direction or Y direction, or multiple aerial operation detection sensors may be arranged at multiple positions in the forward or backward direction.

[0319] In addition, in the example of FIG. 25A, when the mobile terminal 2000b is placed in the display range 3R, the mobile terminal 2000b is reflected in the captured image of the imaging unit 1180 (camera). Therefore, the image control unit 1160 and the like linked to the imaging unit 1180 can detect the mobile terminal 2000b from the captured image and determine whether the mobile terminal 2000b has contacted the display range 3R. As a result, the space-floating image display device 1000 can detect the mobile terminal 2000b as a contact operation when it contacts the display range 3R as shown in FIG. 25A by using the imaging unit 1180. The space-floating image display device 1000 may determine and detect the contact operation using at least one of the aerial operation detection sensor 1351 and the imaging unit 1180. When both of them are used, the accuracy of the determination and detection can be increased.

[0320] As another arrangement of the imaging unit 1180 (camera), other arrangements may be used, such as an arrangement shown by a dashed line as imaging unit 1180b, at the outside of the housing 1190, at a position on the far side in the Y direction, with the optical axis facing the display range 3R. The imaging unit 1180b may capture an image of the face of the user 230, or may detect that the user 230 has approached. In addition, the imaging unit 1180b is not limited to the mid-air operation detection sensor 1351 that senses parallel to the plane of the floating-in-space image 3 as shown in FIG. 25B, but may use a distance measuring sensor or a stereo camera that senses the plane from a direction perpendicular to the plane of the floating-in-space image 3.

[0321] FIG. 25C shows another example of the arrangement of the aerial operation detection sensor 1351 and the imaging unit 1180 (camera) in the housing 1190. The example of FIG. 25C shows a case where the housing 1190 is vertically placed and the floating image 3 is formed to stand vertically (Z direction) based on FIG. 2B as an optical system. The components of the optical system described above are mounted on the rear part of the housing 1190 in the Y direction, and the floating image 3 is formed between the upper and lower housing parts at the front part of the housing 1190. In this example, the aerial operation detection sensor 1351 and the imaging unit 1180 (camera) are installed on the upper housing part. The aerial operation detection sensor 1351 faces downward so that the optical axis overlaps with the display range 3R. The imaging range of the imaging unit 1180 (camera) is set to include the display range 3R. Even with this configuration, when the mobile terminal 2000 is inserted into the display range 3R, a contact operation can be detected using at least one of the mid-air operation detection sensor 1351 and the imaging section 1180 (camera).

[0322] [Distinguishing between fingers and mobile devices] The space floating image display device 1000 may or may not distinguish between the fingers of the user 230 and the mobile terminal 2000 as objects that contact the space floating image 3 (display range 3R). When distinguishing between them, the following technical means may be mentioned, for example.

[0323] (1) The space-floating image display device 1000 may recognize the mobile terminal 2000 from communication information such as a connection request when performing any communication between the mobile terminal 2000 and the space-floating image display device 1000. When the mid-air operation detection sensor 1351 detects that some object is in contact with the display range 3R, if there is the communication information, the space-floating image display device 1000 may estimate that the object is the mobile terminal 2000.

[0324] (2) The space floating image display device 1000 may recognize and determine that the object in contact with the space floating image 3 is the mobile terminal 2000 by performing image recognition processing or the like based on an image captured by the imaging unit 1180 (camera).

[0325] (3) The floating-in-space image display device 1000 may use the mid-air operation detection sensor 1351 and the mid-air operation detection unit 1350 to determine whether the object in contact with the floating-in-space image 3 is the finger of the user 230 or the mobile terminal 2000. For example, as a normal mid-air operation / touch operation, an operation using one finger is accepted (for example, detection of the contact point 2501 in FIG. 25B). When the mobile terminal 2000 is inserted into the display range 3R, the area of ​​the contact point is larger for the mobile terminal 2000 than for a single finger, and the shape of the contact point is linear. The floating-in-space image display device 1000 detects and determines the area and shape of the contact point, for example, the contact point 2503 in FIG. 25B, using the mid-air operation detection sensor 1351, etc., and determines whether the contacting object is a finger or a mobile terminal 2000 based on the result.

[0326] (4) The information exchanged between the space-floating image display device 1000 and the mobile terminal 2000 includes information that represents the mobile terminal 2000, such as a QR code described below. The space-floating image display device 1000 recognizes and acquires this information, thereby distinguishing whether the object touching the space-floating image 3 is the finger of the user 230 or the mobile terminal 2000.

[0327] [Mobile device applications and screens] 26A shows a display example of the screen of the display unit 20012 by the application 2010 of the mobile terminal 2000 in the third embodiment. The mobile terminal 2000 displays an application screen 2601 on the screen of the display unit 20012 by processing of the OS, the application 2010, etc. based on the input operation of the user 230. In this example, the application 2010 is a dedicated app that cooperates with the space-floating image display device 1000 to allow the user 230 to conveniently use various functions of the space-floating image display device 1000.

[0328] The application screen 2601 in FIG. 26A has a GUI that prompts the user 230 to specify an image 2020 (target image) to be displayed on the space-floating image 3 of the space-floating image display device 1000. For example, a message such as "Please specify an image to be displayed on the aerial display" is displayed. The user 230 selects an image 2020 (FIG. 15) to be displayed on the aerial display from images in the mobile terminal 2000. The selected image is previewed as a display target image 2602. After checking the display target image 2602, the user 230 presses a display request button 2603. As a result, the mobile terminal 2000 transmits a display request 2030 with the display target image 2602 as the target image to the space-floating image display device 1000 by short-distance wireless communication or the like.

[0329] Fig. 26B is a modified example of the display of the guide image in Fig. 21A described above, and is an example of displaying a guide image on the screen of the mobile terminal 2000. An application screen 2601 in Fig. 26B displays a guide image 2604 for encouraging the user 230 to perform a contact operation. The guide image 2604 displays a message such as "Please touch the smartphone to the floating image", and an image showing the action of inserting and touching the mobile terminal 2000 to the display range 3R is displayed.

[0330] Fig. 26C is a modified example of the display of the guide image in Fig. 23A and the display of the guide image in Fig. 24A described above, and is an example of displaying a guide image on the screen of the mobile terminal 2000. In the application screen 2601 in Fig. 26C, depending on the situation, a guide image 2605 for informing the user 230 that the display request 2030 is in progress, a guide image 2606 for informing the user 230 that the display request 2030 has been received and succeeded, or the like is displayed.

[0331] In the third embodiment, as one method, during a contact operation, as described above, the user 230 designates the target image 2020 on the screen of the application 2010 of the mobile terminal 2000, presses the display request button 2603, and the mobile terminal 2000 transmits the display request 2030. After that, the user 230 inserts the mobile terminal 2000 into the display range 3R.

[0332] Not limited to this, in other embodiments, the operation of the user 230 may be kept to a minimum. For example, the user 230 displays a desired target image on the screen of the mobile terminal 2000, and in that state inserts the mobile terminal 2000 into the display range 3R. The space-floating image display device 1000 judges and detects the contact operation, and generates a display request 2030 if successful. The space-floating image display device 1000 communicates with the mobile terminal 2000, obtains a target image associated with the display request 2030, and displays it on the space-floating image 3. The target image is the image displayed on the screen of the mobile terminal 2000 at that time.

[0333] [Display of target images in a floating space video] The image control unit 1160 (FIG. 3) of the space floating image display device 1000 generates image data to be displayed on the screen of the display device 1 (liquid crystal display panel 11) based on the image data of the target image. For example, if the target image is two-dimensional image data, this image data is two-dimensional image data generated by processing the two-dimensional image data. Also, if the target image is three-dimensional image data (such as a three-dimensional model), this image data is two-dimensional image data generated by rendering the three-dimensional image data in a virtual three-dimensional space.

[0334] 16, the above function allows the user 230 to display and specify a target image 1601 (an example of a character image) on the mobile terminal 2000, as an image 1602, as a floating-in-space image 3, in response to a touch operation and a display request 2030. This allows the user 230 to enjoy viewing the desired target image 1601 as the image 1602 of the floating-in-space image 3.

[0335] There is no particular limitation on the method for canceling the display of the target image in the floating in space image 3. For example, the display may be automatically cancelled after a certain time has elapsed since the display of the target image started. Also, a display cancel button or the like may be provided on the floating in space image 3 or the screen of the mobile terminal 2000. A display cancel button or the like may be provided on the housing 1190.

[0336] As described above, according to the space floating image display system of the third embodiment, by linking with the mobile terminal 2000, it is possible to provide the user 230 with a service that allows the user 230 to view a desired image displayed as the space floating image 3.

[0337] In addition, the data and information exchanged between the mobile terminal 2000 and the space floating image display device 1000 may not be limited to the image to be displayed, and some data and information for management and control, such as user information, may be exchanged together with the image to be displayed.

[0338] As a variant example, it is also possible to display a background image, character image, etc. in advance as the floating-in-space image 3, and then, from that state, by the user 230 touching the mobile terminal 2000 and making a display request 2030, to superimpose the target image on the background image, etc.

[0339] In the third embodiment, when the mobile terminal 2000 touches the floating-in-space image 3, it is determined that the display request 2030 is received and the target image is displayed, but this is not limiting. In a modified example, it may be determined that the mobile terminal 2000 is successful when the user 230 inserts the mobile terminal 2000 into the floating-in-space image 3 and touches it, then the mobile terminal 2000 passes through the floating-in-space image 3 and enters the space at the back, and is no longer touching the display range 3R (for example, FIG. 19E).

[0340] In addition, the following modification is also possible. The function in the modification is a function to acquire the image displayed as the space-floating image 3 on the mobile terminal 2000 side. The space-floating image display device 1000 displays a character image or the like on the space-floating image 3 in advance (for example, similar to (3) in FIG. 16). When the user 230 wants to acquire the image displayed as the space-floating image 3 on the mobile terminal 2000 side, the user performs a touch operation of the mobile terminal 2000 on the space-floating image 3 as described above. If this touch operation is successful, the space-floating image display device 1000 transmits the image displayed as the space-floating image 3 (the image data corresponding to it) to the mobile terminal 2000. This allows the user 230 to acquire and display the image on the mobile terminal 2000 and enjoy watching it (for example, similar to (1) in FIG. 16).

[0341] In the third embodiment, after the mobile terminal 2000 is brought into contact with the space-floating image 3 to display the target image, the display of the target image continues even if the mobile terminal 2000 is no longer in contact with the space-floating image 3, but this is not limiting. In a modified example, the function may be such that the target image is displayed only while the mobile terminal 2000 is in contact with the space-floating image 3. When the space-floating image display device 1000 detects that the mobile terminal 2000 has left the display range 3R, it cancels the display of the target image on the space-floating image 3.

[0342] <Example 4> Next, an example using a QR code (two-dimensional code) or the like will be described as another embodiment (hereinafter referred to as embodiment 4) with reference to Fig. 27A and subsequent figures. Embodiment 4 is a modified example of embodiment 3. The mobile terminal 2000 has a function of appropriately displaying a QR code on the screen of the display unit 20012. This QR code may be, for example, a code representing the above-mentioned display request 2030.

[0343] In the fourth embodiment, when the mobile terminal 2000 sends a display request 2030 to the space-floating image display device 1000, the following operations and actions may be performed to make the mobile terminal 2000 recognize the space-floating image display device 1000, or to make the space-floating image display device 1000 recognize the mobile terminal 2000. The user 230 performs an operation to link or connect to the space-floating image display device 1000 and to send a display request 2030 from an application 2010 or the like on the mobile terminal 2000 (similar to FIG. 26A, for example). Then, the user 230 performs a contact operation to insert the mobile terminal 2000 into the display range 3R. Alternatively, for example, the user 230 displays a QR code or the like on the screen of the mobile terminal 2000 before insertion, and then performs a contact operation to insert the QR code or the like of the mobile terminal 2000 into the display range 3R in a position that allows the QR code or the like of the mobile terminal 2000 to be read by the imaging unit 1180 (camera) on the space-floating image display device 1000 side.

[0344] FIG. 27A shows an example in which the mobile terminal 2000 displays a QR code screen 2701 on the screen 1902 of the display unit 20012 based on an input operation by the user 230, and displays image information such as a QR code 2702 on the QR code screen 2701.

[0345] FIG. 27B shows an example in which the user 230 displays a QR code on the screen 1902 of the mobile terminal 2000a in a position before insertion into the display range 3R (FIG. 27A) in the YZ plane view, and then inserts the mobile terminal 2000a into the display range 3R from that position, resulting in the mobile terminal 2000b at the contact position. The mobile terminal 2000b in this state has the screen 1902 facing downward and is within the imaging range of the imaging unit 1180 (camera). Therefore, the space-floating image display device 1000 can detect the mobile terminal 2000b from the captured image and can also detect the QR code 2702 displayed on the screen 1902. The space-floating image display device 1000 recognizes the QR code and extracts the data and information described in the QR code. For example, the space-floating image display device 1000 obtains a display request 2030 from the QR code.

[0346] The space-floating image display device 1000 detects the contact of the mobile terminal 2000b with the display range 3R by the mid-air operation detection sensor 1351 or the imaging unit 1180, and acquires the display request 2030 from the QR code of the mobile terminal 2000b. Based on the success of these operations, the space-floating image display device 1000 displays the target image associated with the display request 2030 on the space-floating image 3.

[0347] 27B, the mobile terminal 2000b is in contact with the display range 3R, but this is not limiting, and as described above, the mobile terminal 2000 may enter a non-contact state by entering beyond the display range 3R after contacting the display range 3R. The orientation of the mobile terminal 2000 may be an orientation in which the QR code on the screen 1902 can be detected by the imaging unit 1180 (camera). For example, in the case of the imaging unit 1180b in FIG. 25A, the orientation of the mobile terminal 2000 may be an orientation in which the screen 1902 faces the rear side in the Y direction (for example, it may be the same as FIG. 19E).

[0348] In the fourth embodiment, as described above, a condition for recognizing a QR code is added to the judgment condition for a predetermined contact operation in the third embodiment. In other words, in the fourth embodiment, when the AND condition that a predetermined contact operation is detected and a QR code is recognized is satisfied, the space-floating image display device 1000 receives and permits the display request 2030 and displays a target image on the space-floating image 3.

[0349] In Example 4 (FIG. 27B), user 230 may insert mobile terminal 2000 into display range 3R in a horizontal direction parallel to the upper surface of housing 1190 with screen 1902 facing downward (similar to FIG. 19E). In a modified example, user 230 may insert mobile terminal 2000 into display range 3R with screen 1902 facing downward, and then place screen 1902 in contact with the upper surface of transparent member 100 of housing 1190. Even in these cases, the QR code can be recognized.

[0350] [QR Code] The data and information described in the code such as the QR code / barcode is not limited to the example of the display request 2030, but may include the following, for example. These data and information may be used in combination.

[0351] (1) Connection information: Information for communication connection between the mobile terminal 2000 and the space-floating image display device 1000 in the communication network of Fig. 15. For example, information such as IP address, ID, and password. The communication network here may be the Internet 3010 or LAN 3020, or may be direct communication between the mobile terminal 2000 and the space-floating image display device 1000.

[0352] (2) Terminal ID / User ID: Information such as the ID of the mobile terminal 2000. Alternatively, information such as the ID of the user 230.

[0353] (3) Image ID: Information such as the ID and URL of the target image that the user 230 wants to display.

[0354] When the above connection information is a QR code, the mobile terminal 2000 transmits the connection information to the space-floating image display device 1000 by presenting the QR code, and a wireless connection is established between the communication unit 20020 of the mobile terminal 2000 and the communication unit 1132 of the space-floating image display device 1000 using the connection information.

[0355] When the terminal ID / user ID, etc. are a QR code, the space-floating image display device 1000 can recognize a specific mobile terminal 2000 / user 230. The space-floating image display device 1000 may allow only a specific mobile terminal 2000 / user 230 associated with a specific terminal ID / user ID to be provided with the service of this function.

[0356] When the image ID is a QR code, the target image is not limited to an image held in the mobile terminal 2000 and can be used.

[0357] As described above, according to the space-floating image display system of the fourth embodiment, by using the QR code, a specific contact operation between the mobile terminal 2000 and the space-floating image display device 1000 can be defined, and various controls can be performed using the QR code.

[0358] Incidentally, even in the case of the embodiment 3 or the like in which the QR code is not used, the above-mentioned data and information may be exchanged between the mobile terminal 2000 and the space floating image display device 1000 together with the display request 2030 and used.

[0359] As a modification of the fourth embodiment, the space floating image display device 1000 may transmit the information of the QR code to the nearby mobile terminal 2000, and the mobile terminal 2000 that receives the information may display the QR code.

[0360] In the third and fourth embodiments, the data of the target image to be displayed on the space-floating image 3 is held in advance in the mobile terminal 2000, but this is not limiting. In a modified example, the data of the target image may be held on the space-floating image display device 1000 side, and the user 230 and the mobile terminal 2000 side may select and designate the target image and transmit the display request 2030. In another modified example, the data of the target image may be held in a server or the like on a communication network, and the user 230 and the mobile terminal 2000 side may select and designate the target image and transmit the display request 2030.

[0361] Also, as a modified example of the fourth embodiment, the space-floating image display device 1000 may display the above-mentioned QR code information on the space-floating image 3, and the user 230 may recognize the QR code information displayed as the space-floating image 3 by the camera of the mobile terminal 2000.

[0362] Fig. 28 (Fig. 28A, etc.) shows this modified example. Fig. 28A is an example in which the space-floating image display device 1000 displays a QR code 2801, which is a space-floating image 3. Fig. 28B is an example in which the mobile terminal 2000 captures and reads the QR code 2801 of the space-floating image 3 in Fig. 28A using the camera of the imaging unit 20025, for example, the outer camera, in a YZ plan view. The mobile terminal 2000 obtains predetermined information from the recognized QR code 2801. After recognizing the QR code 2801, the user 230 brings the mobile terminal 2000 into contact with the display range 3R.

[0363] 28C is a further modified example, which is an example of displaying a QR code 2802 on the screen of the second display device 1680 of the space-floating image display device 1000. The space-floating image display device 1000 uses the second display device 1680 arranged so as to overlap the space-floating image 3 on the rear side, as in the case of FIG. 4M, for example, to display the QR code 2802 on the screen of the second display device 1680. Also, the space-floating image display device 1000 may use the transmissive self-luminous image display device 1650 to display a QR code on the screen of the transmissive self-luminous image display device 1650, as in the case of FIG. 4L.

[0364] User 230 captures an image of QR code 2802 displayed on the screen of second display device 1680 with the camera of imaging unit 20025 of portable terminal 2000, and portable terminal 2000 recognizes the QR code and obtains information. After recognizing QR code 2802, user 230 brings portable terminal 2000 into contact with display range 3R.

[0365] The technology according to this embodiment displays high-resolution, high-brightness image information in a state of floating in space, which allows users to operate the system without worrying about contact infection. If the technology according to this embodiment is used in a system used by an unspecified number of users, it is possible to provide a contactless user interface that can be used without worry, reducing the risk of contact infection. This contributes to the achievement of "Good health and well-being for all," one of the Sustainable Development Goals (SDGs) advocated by the United Nations.

[0366] In addition, the technology according to this embodiment makes it possible to efficiently reflect only the normal reflected light from the retroreflector by reducing the divergence angle of the emitted image light and aligning it to a specific polarization, which makes it possible to obtain a bright and clear floating image with high light utilization efficiency. The technology according to this embodiment makes it possible to provide a highly usable non-contact user interface that can significantly reduce power consumption. This contributes to the achievement of the Sustainable Development Goals (SDGs) proposed by the United Nations, "9 Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote technological innovation" and "11 Make cities and towns sustainable and livable."

[0367] Although various embodiments have been described above in detail, the present invention is not limited to the above-described embodiments, and various modified examples are included. For example, the above-described embodiments are detailed descriptions of the entire system in order to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]

[0368] 1...display device, 2...retroreflector, 3...space-floating image (floating image), 3R...display range, 100...transparent member, 101, 101B...polarized light separation member, 11...liquid crystal display panel, 12...absorptive polarizer, 13...light source device, 230...user, 1000...space-floating image display device, 1110...control unit, 1160...image control unit, 1180...imaging unit, 1200...opening, 1350...mid-air operation detection unit, 1351...mid-air operation detection sensor, 1600...insertion / touch operation, 1601...image, 1602...image, 2000...mobile terminal, 2030...display request, 3000...system.

Claims

1. A floating-in-the-air image display device, An image processing section; a display unit that displays the image that has been processed by the image processing unit; an optical system for generating a floating image based on the image displayed by the display unit; A detection unit that detects contact of an object with a display range of the floating-in-the-air image; A communication unit that communicates with a user's mobile terminal; Equipped with When the touch of the mobile terminal with the display range is detected, a target image designated on the mobile terminal side is displayed as the floating-in-the-air image. A floating image display device.

2. 2. The airborne image display device according to claim 1, a display request is generated from the portable terminal to the floating-in-the-air image display device before, during, or after the portable terminal comes into contact with the display range; when contact of the mobile device with the display range is detected, the display request is received, and a target image specified on the mobile device side is acquired from the mobile device or an external device via communication, and displayed as the floating-in-the-air image. A floating image display device.

3. 2. The airborne image display device according to claim 1, When a contact of the mobile terminal with a specific area of ​​the display range is detected, a target image designated on the mobile terminal side is displayed as the floating-in-the-air image. A floating image display device.

4. 2. The airborne image display device according to claim 1, when a specific attitude of the portable terminal is detected when the portable terminal is in contact with the display range, a target image designated on the portable terminal side is displayed as the floating-in-the-air image; A floating image display device.

5. 2. The airborne image display device according to claim 1, when it is detected that the mobile terminal has entered a space behind the display range after the mobile terminal has come into contact with the display range, a target image designated on the mobile terminal side is displayed as the floating-in-the-air image; A floating image display device.

6. 2. The airborne image display device according to claim 1, When the mobile terminal touches the display range and the touch time exceeds a certain period of time, a target image designated on the mobile terminal side is displayed as the floating-in-the-air image. A floating image display device.

7. 2. The airborne image display device according to claim 1, a guide image for encouraging the user to touch the display area of ​​the mobile device is displayed as the floating-in-space image; A floating image display device.

8. 2. The airborne image display device according to claim 1, displaying, on a screen of the mobile device, a guide image for prompting the user to touch the display range of the mobile device; A floating image display device.

9. 2. The airborne image display device according to claim 1, When detecting an invisible area caused by the contact of the mobile terminal with the display range, a non-display area including the invisible area is determined within the display range, and the non-display area is displayed as the floating image in space in a display area other than the non-display area. A floating image display device.

10. 2. The airborne image display device according to claim 1, The detection unit is at least one of an aerial operation detection sensor that detects an aerial operation in a range including the display range, or an imaging unit that images a range including the display range. A floating image display device.

11. 2. The airborne image display device according to claim 1, Recognizing a code displayed on the screen of the mobile device to obtain information; When the code is recognized and contact of the mobile terminal with the display area is detected, a target image designated on the mobile terminal side is displayed as the floating-in-the-air image. A floating image display device.

12. 2. The airborne image display device according to claim 1, The mobile device recognizes the code displayed on the floating image and acquires information; When the code is recognized and contact of the mobile terminal with the display area is detected, a target image designated on the mobile terminal side is displayed as the floating-in-the-air image. A floating image display device.

13. In the air-floating image display device according to claim 11 or 12, The information in the code is at least one of the following: A display request from the mobile terminal to the airborne image display device; Connection information for communication between the mobile terminal and the airborne image display device; The ID of the mobile terminal or the ID of the user, The ID or URL of the target image; A floating image display device.

14. 2. The airborne image display device according to claim 1, When a contact of the mobile terminal with the display range is detected, the image displayed on the floating-in-space image is acquired by the mobile terminal. A floating image display device.

15. A floating-in-the-air image display system having a floating-in-the-air image display device and a user's mobile terminal, The airborne image display device includes: An image processing section; a display unit that displays the image that has been processed by the image processing unit; an optical system for generating a floating image based on the image displayed by the display unit; A detection unit that detects contact of an object with a display range of the floating-in-the-air image; A communication unit that communicates with a user's mobile terminal; Equipped with When the floating-in-the-air image display device detects contact of the mobile terminal with the display range, the floating-in-the-air image display device displays a target image designated on the mobile terminal side as the floating-in-the-air image. A floating video display system.

16. A portable terminal in a floating-in-the-air image display system having a floating-in-the-air image display device and a user's portable terminal, The airborne image display device includes: An image processing section; a display unit that displays the image that has been processed by the image processing unit; an optical system for generating a floating image based on the image displayed by the display unit; A detection unit that detects contact of an object with a display range of the floating-in-the-air image; A communication unit that communicates with the mobile terminal of the user; Equipped with When the floating-in-the-air image display device detects contact of the mobile terminal with the display range, the floating-in-the-air image display device displays a target image designated by the mobile terminal as the floating-in-the-air image; The mobile terminal designates the target image based on an operation of the user, and transmits image data or information of the designated target image to the air-floating image display device. Mobile device.

Images