Floating image display device

Abstract

This air floating image display device is provided with a display unit (1) for displaying an image, a first polarization separation member (101D), a second polarization separation member (101E), and one or more retroreflection plates (2). A first image display region (1501) and a second image display region (1502) are present in a display screen of the display unit (1). Image light emitted from the first image display region (1501) forms a first air floating image (3D) in the air after being transmitted through the first polarization separation member (101D), and retroreflected by any one retroreflection plate included in the one or more retroreflection plates (2), image light emitted from the second image display region (1502) forms a second air floating image (3E) in the air after being transmitted through or reflected by the second polarization separation member (101E), and retroreflected by any one retroreflection plate included in the one or more retroreflection plates (2), and consequently, the first air floating image (3D) and the second air floating image (3E) form an air floating image with multiple layers having different depth directions when viewed from a user.

Description

[Technical Field]

[0001] The present invention relates to a floating-in-the-air image display device. [Background technology]

[0002] The floating information display technology is disclosed in, for example, Patent Document 1. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-128722 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the disclosure of Patent Document 1 does not sufficiently consider a configuration for obtaining practical brightness and quality for the floating image, or a configuration for allowing the user to view the floating image more enjoyably.

[0005] An object of the present invention is to provide a more suitable floating-in-the-air image display device. [Means for solving the problem]

[0006] In order to solve the above problem, for example, the configuration described in the claims is adopted. The present application includes a plurality of means for solving the above problem, and one example thereof is a floating-in-the-air image display device that displays floating images, comprising a display unit that displays images, a first polarization separating member, a second polarization separating member, and one or more retroreflectors, and the display screen of the display unit has a first image display area and a second image display area, and image light emitted from the first image display area of ​​the display screen of the display unit is transmitted through the first polarization separating member and reflected by any one of the one or more retroreflectors. The image light emitted from the second image display area of ​​the display screen of the display unit is transmitted through or reflected by the second polarization separation member and is retroreflected by any one of the one or more retroreflectors, and then a second floating image is formed in the air, so that the first floating image and the second floating image form a multi-layer floating image with different depth directions as seen by the user. [Effects of the Invention]

[0007] According to the present invention, a more suitable floating-in-the-air image display device can be realized. Other problems, configurations, and effects will become clear in the following description of the embodiments. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram showing an example of a usage form of a space floating image display device according to an embodiment of the present invention; [Figure 2A] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 2B] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 2C] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 3]1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4A] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4B] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4C] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4D] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4E] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4F] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4G] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4H] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4I] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4J] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4K] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4L] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4M] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 5] 1 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. [Figure 6] 1 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. [Figure 7]1 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. [Figure 8] 1 is a layout diagram showing a main part of a space floating image display device according to an embodiment of the present invention; [Figure 9] 1 is a cross-sectional view showing a configuration of a display device according to an embodiment of the present invention. [Figure 10] 1 is a cross-sectional view showing a configuration of a display device according to an embodiment of the present invention. [Figure 11] 1 is an explanatory diagram for explaining the light source diffusion characteristics of an image display device according to an embodiment of the present invention. [Figure 12] 1 is an explanatory diagram for explaining the diffusion characteristics of a video display device according to an embodiment of the present invention; [Figure 13A] 1 is a diagram illustrating an example of a problem to be solved by image processing according to an embodiment of the present invention; [Figure 13B] FIG. 10 is an explanatory diagram of an example of image processing according to an embodiment of the present invention. [Figure 13C] FIG. 10 is an explanatory diagram of an example of a video display process according to an embodiment of the present invention. [Figure 13D] FIG. 10 is an explanatory diagram of an example of a video display process according to an embodiment of the present invention. [Figure 14] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 15A] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 15B] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 16A] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 16B] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 17A] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 17B] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 17C] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 17D] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 18A] 1 is a diagram showing an example of a display of a space floating image display device according to an embodiment of the present invention; [Figure 18B] 1 is a diagram showing an example of a display of a space floating image display device according to an embodiment of the present invention; [Figure 18C] 1 is a diagram showing an example of a display of a space floating image display device according to an embodiment of the present invention; [Figure 18D] 1 is a diagram showing an example of a display of a space floating image display device according to an embodiment of the present invention; [Figure 18E] 1 is a diagram showing an example of a display of a space floating image display device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the description of the embodiments, and various changes and modifications can be made by those skilled in the art within the scope of the technical ideas disclosed in this specification. Furthermore, in all drawings used to explain the present invention, components having the same functions are given the same reference numerals, and repeated explanations thereof may be omitted.

[0010] The following examples relate to an image display device that can transmit an image generated by image light from an image light source through a transparent member that separates a space, such as glass, and display the image as a floating image outside the transparent member. In the following explanation of the examples, the image floating in space is expressed using the term "floating image in space." Instead of this term, it is also acceptable to express it as "aerial image," "spatial image," "floating image in space," "floating optical image of displayed image," "floating optical image of displayed image," etc. The term "floating image in space," which is mainly used in the explanation of the examples, is used as a representative example of these terms.

[0011] According to the following embodiments, an image display device suitable for, for example, bank ATMs, train station ticket machines, digital signage, and the like can be realized. For example, currently, bank ATMs, train station ticket machines, and the like typically use touch panels. However, by using a transparent glass surface or a light-transmitting plate, high-resolution image information can be displayed in a floating state on the glass surface or light-transmitting plate. In this case, by making the divergence angle of the emitted image light small, i.e., an acute angle, and further aligning it with a specific polarization, only the normal reflected light is efficiently reflected by the retroreflector. This improves light utilization efficiency and suppresses the ghost images that occur in addition to the main floating image, which is a problem with conventional retroreflection systems, thereby achieving a clear floating image. Furthermore, by using a device including the light source of this embodiment, a novel and highly usable floating image display device (floating image display system) can be provided that can significantly reduce power consumption. Furthermore, a floating image display device for a vehicle can be provided that can display a so-called unidirectional floating image that can be viewed inside and / or outside the vehicle. Example 1

[0012] <Example of how to use the space floating image display device> FIG. 1 is a diagram showing an example of a usage form of a space-floating image display device according to an embodiment of the present invention, and is a diagram showing the overall configuration of the space-floating image display device according to this embodiment. The specific configuration of the space-floating image display device will be described in detail using FIG. 2 and other figures. Light with a narrow-angle directional characteristic and specific polarization is emitted from image display device 1 as an image light beam, reflected by the optical system within the space-floating image display device, and then incident on retroreflector 2. It is then retroreflected and transmitted through transparent member 100 (glass, etc.), forming a real aerial image (space-floating image 3) on the outside of the glass surface. In the following embodiments, the retroreflector 2 (retroreflector) is used as an example of a retroreflector. However, the retroreflector 2 of the present invention is not limited to a planar plate, and is used as an example of a concept including a sheet-like retroreflector attached to a planar or non-planar member, or an entire assembly in which a sheet-like retroreflector is attached to a planar or non-planar member.

[0013] In addition, in a store or the like, a space is partitioned by a show window (also called "window glass") 105, which is a translucent member such as glass. According to the space floating image display device of this embodiment, it is possible to transmit such a transparent member and display a floating image in one direction to the outside and / or inside of the store (space).

[0014] 1, the inside of the window glass 105 (inside the store) is shown in the depth direction, and the outside (for example, the sidewalk) is shown in the foreground. On the other hand, by providing a means for reflecting specific polarized waves on the window glass 105, it is possible to reflect the waves and form an aerial image at a desired position inside the store.

[0015] <Configuration example of optical system for space floating image display device> 2A is a diagram showing an example of the configuration of an optical system of a space-floating image display device according to one embodiment of the present invention. The configuration of the space-floating image display device will be described in more detail using FIG. 2A. As shown in FIG. 2A(1), a display device 1 that diverges specific polarized image light at a narrow angle is provided in an oblique direction of a transparent member 100 such as glass. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates specific polarized light with narrow-angle diffusion characteristics.

[0016] Image light of a specific polarization from the display device 1 is reflected by a polarization separator 101 (in the figure, the polarization separator 101 is formed into a sheet and adhered to the transparent member 100) that has a film that selectively reflects image light of a specific polarization and is provided on a transparent member 100, and then enters the retroreflector 2. A λ / 4 plate 21 is provided on the image light incident surface of the retroreflector 2. The image light passes through the λ / 4 plate 21 twice, once when it enters the retroreflector 2 and once when it exits, thereby undergoing polarization conversion from the specific polarization to the other polarization. Here, the polarization separator 101 that selectively reflects image light of a specific polarization has the property of transmitting the polarized light of the other polarization that has been polarization-converted, so the image light of the specific polarization after polarization conversion passes through the polarization separator 101. The image light that has passed through the polarization separator 101 forms a space-floating image 3, which is a real image, outside the transparent member 100.

[0017] Here, a first example of polarization design for the optical system of FIG. 2A will be described. For example, S-polarized image light may be emitted from display device 1 to polarization separator 101, which may have the property of reflecting S-polarized light and transmitting P-polarized light. In this case, the S-polarized image light reaching polarization separator 101 from display device 1 is reflected by polarization separator 101 and travels toward retroreflector 2. When the image light is reflected by retroreflector 2, it passes twice through λ / 4 plate 21 provided on the incident surface of retroreflector 2, converting the image light from S-polarized light to P-polarized light. The P-polarized image light then travels back toward polarization separator 101. Here, polarization separator 101 has the property of reflecting S-polarized light and transmitting P-polarized light, so the P-polarized image light passes through polarization separator 101 and then through transparent member 100. The image light transmitted through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101. This polarization design allows the floating image 3 to be formed optimally.

[0018] Next, a second example of polarization design for the optical system of FIG. 2A will be described. For example, a configuration may be adopted in which P-polarized image light is emitted from display device 1 to polarization separator 101, and polarization separator 101 has the property of reflecting P-polarized light and transmitting S-polarized light. In this case, the P-polarized image light that reaches polarization separator 101 from display device 1 is reflected by polarization separator 101 and travels toward retroreflector 2. When the image light is reflected by retroreflector 2, it passes twice through λ / 4 plate 21 provided on the incident surface of retroreflector 2, thereby converting the P-polarized image light to S-polarized light. The S-polarized image light then travels back toward polarization separator 101. Here, polarization separator 101 has the property of reflecting P-polarized light and transmitting S-polarized light, so the S-polarized image light passes through polarization separator 101 and then through transparent member 100. The image light transmitted through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101. This polarization design allows the floating image 3 to be formed optimally.

[0019] The light that forms the floating image 3 is a collection of light rays that converge from the retroreflector 2 to the optical image of the floating image 3, and these light rays continue to travel in a straight line even after passing through the optical image of the floating image 3. Therefore, the floating image 3 is a highly directional image, unlike the diffused image light formed on a screen by a general projector or the like. Therefore, in the configuration of FIG. 2A, when a user views the floating image 3 from the direction of arrow A, the floating image 3 is perceived as a bright image. However, when another person views the floating image 3 from the direction of arrow B, the floating image 3 cannot be perceived as an image at all. This characteristic is very suitable for use in a system that displays images that require high security or highly confidential images that should be kept secret from people directly facing the user.

[0020] Depending on the performance of the retroreflector 2, the polarization axis of the reflected image light may become irregular. The reflection angle may also become irregular. Such irregular light may not maintain the polarization state and propagation angle assumed in the design. For example, light with an unintended polarization state and propagation angle may re-enter the image display surface of the liquid crystal display panel 11 directly from the position of the retroreflector 2 without passing through a polarization separation member. Such light with an unintended polarization state and propagation angle may re-enter the image display surface of the liquid crystal display panel 11 after being reflected by components within the space-floating image display device. Such light re-entering the image display surface of the liquid crystal display panel 11 may be re-reflected by the image display surface of the liquid crystal display panel 11 constituting the display device 1, potentially generating ghost images and degrading the image quality of the space-floating image. Therefore, in this embodiment, an absorbing polarizer 12 may be provided on the image display surface of the display device 1. The image light emitted from the display device 1 is transmitted through the absorptive polarizer 12, and the reflected light returning from the polarization separation member 101 is absorbed by the absorptive polarizer 12, thereby suppressing the re-reflection. This makes it possible to prevent degradation of image quality due to ghost images of spatially floating images. Specifically, if the display device 1 is configured to emit S-polarized image light to the polarization separation member 101, the absorptive polarizer 12 may be a polarizer that absorbs P-polarized light. Furthermore, if the display device 1 is configured to emit P-polarized image light to the polarization separation member 101, the absorptive polarizer 12 may be a polarizer that absorbs S-polarized light.

[0021] The polarization separation member 101 may be formed of, for example, a reflective polarizing plate or a metal multilayer film that reflects specific polarized waves.

[0022] Next, Figure 2A(2) shows the surface shape of a retroreflector manufactured by Nippon Carbide Industries Co., Ltd., which was used in this study as a representative retroreflector 2. Light rays incident on the regularly arranged hexagonal prisms are reflected by the walls and bottoms of the hexagonal prisms and emitted as retroreflected light in a direction corresponding to the incident light, and a real floating image is displayed based on the image displayed on the display device 1.

[0023] The resolution of this floating image in space depends not only on the resolution of the liquid crystal display panel 11, but also on the outer diameter D and pitch P of the retroreflective portion of the retroreflector 2 shown in Figure 2A(2). For example, when using a 7-inch WUXGA (1920 x 1200 pixels) liquid crystal display panel, even if one pixel (one triplet) is approximately 80 μm, if the diameter D of the retroreflective portion is 240 μm and the pitch is 300 μm, one pixel of the floating image in space will be equivalent to 300 μm. As a result, the effective resolution of the floating image in space will be reduced to about one-third.

[0024] Therefore, in order to make the resolution of the spatial floating image equivalent to that of the display device 1, it is desirable to make the diameter and pitch of the retroreflective portion close to one pixel of the liquid crystal display panel. On the other hand, to suppress the occurrence of moire caused by the retroreflective plate and the pixels of the liquid crystal display panel, it is advisable to design the pitch ratio of each to be a different integer multiple of one pixel. Also, it is advisable to arrange the shape so that none of the sides of the retroreflective portion overlaps with any of the sides of one pixel of the liquid crystal display panel.

[0025] The surface shape of the retroreflector according to this embodiment is not limited to the above example. Various surface shapes that achieve retroreflection may be used. Specifically, the surface of the retroreflector according to this embodiment may be provided with retroreflection elements in which triangular pyramidal prisms, hexagonal pyramidal prisms, other polygonal prisms, or a combination of these are periodically arranged. Alternatively, the surface of the retroreflector according to this embodiment may be provided with retroreflection elements in which these prisms are periodically arranged to form cube corners. Alternatively, the surface of the retroreflector according to this embodiment may be provided with capsule lens-type retroreflection elements in which glass beads are periodically arranged. The detailed configuration of these retroreflection elements can be achieved using existing technology, so a detailed description will be omitted. Specifically, the techniques disclosed in Japanese Patent Laid-Open Nos. 2001-33609, 2001-264525, 2005-181555, 2008-70898, and 2009-229942 may be used.

[0026] <Another configuration example 1 of the optical system of the space floating image display device> Another example of the configuration of the optical system of the space floating image display device will be explained using Fig. 2B. In Fig. 2B, components with the same reference numerals as Fig. 2A have the same functions and configurations as Fig. 2A. For the sake of simplicity, repeated explanations of such components will be omitted.

[0027] In the optical system of FIG. 2B, as in FIG. 2A, image light of a specific polarization is output from the display device 1. The image light of a specific polarization output from the display device 1 is input to a polarization separator 101B. The polarization separator 101B is a member that selectively transmits image light of a specific polarization. Unlike the polarization separator 101 of FIG. 2A, the polarization separator 101B is not integrated with the transparent member 100 but has an independent plate-like shape. Therefore, the polarization separator 101B may also be referred to as a polarization separator plate. The polarization separator 101B may be configured as a reflective polarizer configured by attaching a polarization separator sheet to a transparent member. Alternatively, the transparent member may be formed of a metal multilayer film that selectively transmits specific polarization and reflects polarization of other specific polarizations. In FIG. 2B, the polarization separator 101B is configured to transmit image light of a specific polarization output from the display device 1.

[0028] The image light that has passed through the polarization separation member 101B is incident on the retroreflector 2. A λ / 4 plate 21 is provided on the image light incident surface of the retroreflector. The image light is polarized and converted from a specific polarization to the other polarization by passing through the λ / 4 plate 21 twice, once when it enters the retroreflector and once when it leaves. Here, the polarization separation member 101B has the property of reflecting the polarized light of the other polarization that has been polarized and converted by the λ / 4 plate 21, so the image light after polarization conversion is reflected by the polarization separation member 101B. The image light reflected by the polarization separation member 101B passes through the transparent member 100 and forms a space-floating image 3, which is a real image, outside the transparent member 100.

[0029] Here, a first example of polarization design for the optical system of FIG. 2B will be described. For example, a configuration may be adopted in which P-polarized image light is emitted from display device 1 to polarization separator 101B, and polarization separator 101B has the property of reflecting S-polarized light and transmitting P-polarized light. In this case, the P-polarized image light that reaches polarization separator 101B from display device 1 passes through polarization separator 101B and proceeds to retroreflector 2. When the image light is reflected by retroreflector 2, it passes twice through λ / 4 plate 21 provided on the incident surface of retroreflector 2, thereby converting the P-polarized image light to S-polarized light. The S-polarized image light then proceeds again to polarization separator 101B. Here, polarization separator 101B has the property of reflecting S-polarized light and transmitting P-polarized light, so the S-polarized image light is reflected by polarization separator 101 and passes through transparent member 100. The image light transmitted through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a space-floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101B. This polarization design allows the space-floating image 3 to be formed optimally.

[0030] Next, a second example of polarization design for the optical system of FIG. 2B will be described. For example, S-polarized image light may be emitted from display device 1 to polarization separator 101B, which may have the property of reflecting P-polarized light and transmitting S-polarized light. In this case, the S-polarized image light reaching polarization separator 101B from display device 1 passes through polarization separator 101B and proceeds to retroreflector 2. When the image light is reflected by retroreflector 2, it passes twice through λ / 4 plate 21 provided on the incident surface of retroreflector 2, converting the image light from S-polarized light to P-polarized light. The P-polarized image light then proceeds again to polarization separator 101B. Here, polarization separator 101B has the property of reflecting P-polarized light and transmitting S-polarized light, so the P-polarized image light is reflected by polarization separator 101 and passes through transparent member 100. The image light transmitted through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a space-floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101B. This polarization design allows the space-floating image 3 to be formed optimally.

[0031] In FIG. 2B , the image display surface of the display device 1 and the surface of the retroreflector 2 are arranged parallel to each other. The polarization separator 101B is arranged tilted at an angle α (e.g., 30°) relative to the image display surface of the display device 1 and the surface of the retroreflector 2. When the polarization separator 101B reflects the image light, the direction of the image light reflected by the polarization separator 101B (the direction of the chief ray of the image light) differs by an angle β (e.g., 60°) from the direction of the image light incident from the retroreflector 2 (the direction of the chief ray of the image light). With this configuration, the optical system of FIG. 2B outputs the image light toward the outside of the transparent member 100 at a predetermined angle shown in the figure, forming the space-floating image 3, which is a real image. In the configuration of FIG. 2B , when a user views the space-floating image 3 from the direction of arrow A, the space-floating image 3 is perceived as a bright image. However, when another person views the space-floating image 3 from the direction of arrow B, the space-floating image 3 cannot be perceived as an image at all. This characteristic is extremely suitable for use in a system that displays images that require high security or highly confidential images that should be concealed from people directly facing the user.

[0032] As described above, the optical system of FIG. 2B has a different configuration from the optical system of FIG. 2A, but can form a suitable floating image in space, similar to the optical system of FIG. 2A.

[0033] An absorptive polarizing plate may be provided on the surface of the transparent member 100 facing the polarization separation member 101B. This absorptive polarizing plate may transmit the polarized waves of the image light from the polarization separation member 101B and absorb the polarized waves that are 90° out of phase with the polarized waves of the image light from the polarization separation member 101B. In this way, the image light for forming the space-floating image 3 can be sufficiently transmitted while reducing the external light incident on the space-floating image 3 side of the transparent member 100 by approximately 50%. This makes it possible to reduce stray light in the optical system of FIG. 2B due to the external light incident on the space-floating image 3 side of the transparent member 100.

[0034] <Another configuration example 2 of the optical system of the space floating image display device> Another example of the configuration of the optical system of the space floating image display device will be explained using Fig. 2C. In Fig. 2C, components with the same reference numerals as Fig. 2B have the same functions and configurations as Fig. 2B. For the sake of simplicity, such components will not be described repeatedly.

[0035] The only difference between the optical system in Figure 2B and the optical system in Figure 2C is the angle at which the polarization separation member 101B is disposed relative to the image display surface of the display device 1 and the surface of the retroreflector 2. All other configurations are the same as those of the optical system in Figure 2B, so repeated explanations will be omitted. The polarization design of the optical system in Figure 2C is also the same as that of the optical system in Figure 2B, so repeated explanations will be omitted.

[0036] In the optical system of FIG. 2C , the polarization separator 101B is tilted at an angle α with respect to the image display surface of the display device 1 and the surface of the retroreflector 2. In FIG. 2C , the angle α is 45°. With this configuration, when the polarization separator 101B reflects, the angle β between the direction of propagation of the image light incident from the retroreflector 2 (the direction of the chief ray of the image light) and the direction of propagation of the image light reflected by the polarization separator 101B (the direction of the chief ray of the image light) is 90°. With this configuration, the image display surface of the display device 1 and the surface of the retroreflector 2 are perpendicular to the direction of propagation of the image light reflected by the polarization separator 101B, simplifying the angular relationships of the surfaces that make up the optical system. By arranging the surface of the transparent member 100 so that it is perpendicular to the direction of propagation of the image light reflected by the polarization separator 101B, the angular relationships of the surfaces that make up the optical system can be further simplified. In the configuration of Figure 2C, when a user views the floating image 3 from the direction of arrow A, the floating image 3 is perceived as a bright image. However, when another person views the floating image 3 from the direction of arrow B, the floating image 3 cannot be seen as an image at all. This characteristic is very suitable for use in a system that displays images that require high security or highly confidential images that should be kept secret from people directly facing the user.

[0037] As described above, the optical system of Fig. 2C has a different configuration from the optical systems of Fig. 2A and Fig. 2B, but can form a suitable floating image in space similar to the optical systems of Fig. 2A and Fig. 2B. In addition, the angles of the surfaces constituting the optical system can be made simpler.

[0038] An absorptive polarizer may be provided on the surface of the transparent member 100 facing the polarization separation member 101B. This absorptive polarizer may transmit the polarized waves of the image light from the polarization separation member 101B and absorb the polarized waves that are 90° out of phase with the polarized waves of the image light from the polarization separation member 101B. This allows the image light for forming the space-floating image 3 to be sufficiently transmitted while reducing the external light incident on the space-floating image 3 side of the transparent member 100 by approximately 50%. This allows the stray light in the optical system of FIG. 2C due to the external light incident on the space-floating image 3 side of the transparent member 100 to be reduced.

[0039] According to the optical system of FIGS. 2A, 2B, and 2C described above, it is possible to provide a brighter, higher quality floating image in space.

[0040] <<Block diagram of the internal configuration of the space floating image display device>>

[0041] Next, a description will be given of a block diagram of the internal configuration of the space-floating image display device 1000. Fig. 3 is a block diagram showing an example of the internal configuration of the space-floating image display device 1000.

[0042] The space-floating image display device 1000 includes a retroreflection unit 1101, an image display unit 1102, a light guide 1104, a light source 1105, a power supply 1106, an external power supply input interface 1111, an operation input unit 1107, a nonvolatile memory 1108, a memory 1109, a control unit 1110, a video signal input unit 1131, an audio signal input unit 1133, a communication unit 1132, an aerial operation detection sensor 1351, an aerial operation detection unit 1350, an audio output unit 1140, an image control unit 1160, a storage unit 1170, an imaging unit 1180, etc. In addition, the space-floating image display device 1000 may also include a removable media interface 1134, an attitude sensor 1113, a transmissive self-luminous image display device 1650, a second display device 1680, or a secondary battery 1112.

[0043] Each component of the space floating image display device 1000 is disposed in a housing 1190. The imaging unit 1180 and the mid-air operation detection sensor 1351 shown in FIG.

[0044] The retroreflecting portion 1101 in Fig. 3 corresponds to the retroreflector 2 in Fig. 2A, Fig. 2B, and Fig. 2C. The retroreflecting portion 1101 retroreflects light modulated by the image display portion 1102. Of the light reflected from the retroreflecting portion 1101, the light output to the outside of the space-floating image display device 1000 forms the space-floating image 3.

[0045] 3 corresponds to the liquid crystal display panel 11 in FIGS. 2A, 2B, and 2C. The light source 1105 in FIG. 3 corresponds to the light source device 13 in FIGS. 2A, 2B, and 2C. The image display unit 1102, the light guide 1104, and the light source 1105 in FIG. 3 correspond to the display device 1 in FIGS. 2A, 2B, and 2C.

[0046] The video display unit 1102 is a display unit that generates a video by modulating transmitted light based on a video signal input under the control of a video control unit 1160 (described later). The video display unit 1102 corresponds to the liquid crystal display panel 11 in FIGS. 2A, 2B, and 2C. For example, a transmissive liquid crystal panel is used as the video display unit 1102. Alternatively, for example, a reflective liquid crystal panel that modulates reflected light or a DMD (Digital Microscope) may be used as the video display unit 1102. A Micro Device (registered trademark) panel or the like may also be used.

[0047] The light source 1105 generates light for the image display unit 1102 and is a solid-state light source such as an LED light source or a laser light source. The power source 1106 converts AC current input from the outside via the external power input interface 1111 into DC current and supplies power to the light source 1105. The power source 1106 also supplies the necessary DC current to each unit within the space-floating image display device 1000. The secondary battery 1112 stores the power supplied from the power source 1106. The secondary battery 1112 also supplies power to the light source 1105 and other components that require power via the external power input interface 1111 when power is not supplied from the outside. In other words, when the space-floating image display device 1000 is equipped with the secondary battery 1112, the user can use the space-floating image display device 1000 even when power is not supplied from the outside.

[0048] The light guide 1104 guides light generated by the light source 1105 and irradiates it onto the video display unit 1102. The combination of the light guide 1104 and the light source 1105 can also be called a backlight for the video display unit 1102. The light guide 1104 may be configured mainly using glass. The light guide 1104 may be configured mainly using plastic. The light guide 1104 may be configured using a mirror. There are various possible combinations of the light guide 1104 and the light source 1105. Specific configuration examples of the combination of the light guide 1104 and the light source 1105 will be described in detail later.

[0049] The aerial operation detection sensor 1351 is a sensor that detects an operation on the floating in space image 3 by the finger of the user 230. The aerial operation detection sensor 1351 senses, for example, an area that overlaps with the entire display area of ​​the floating in space image 3. Note that the aerial operation detection sensor 1351 may only sense an area that overlaps with at least a portion of the display area of ​​the floating in space image 3.

[0050] Specific examples of the aerial operation detection sensor 1351 include a distance sensor that uses invisible light such as infrared light, an invisible laser, ultrasonic waves, etc. The aerial operation detection sensor 1351 may also be configured to detect coordinates on a two-dimensional plane by combining multiple sensors. The aerial operation detection sensor 1351 may also be configured with a ToF (Time of Flight) LiDAR (Light Detection and Ranging) or an image sensor.

[0051] The mid-air operation detection sensor 1351 only needs to be capable of sensing to detect touch operations, etc., made by the user with their finger on an object displayed as the floating-in-space image 3. Such sensing can be performed using existing technology.

[0052] The aerial operation detection unit 1350 acquires a sensing signal from the aerial operation detection sensor 1351, and based on the sensing signal, determines whether or not the finger of the user 230 has made contact with an object in the floating in space image 3, and calculates the position (contact position) where the finger of the user 230 has made contact with the object. The aerial operation detection unit 1350 is configured with a circuit such as an FPGA (Field Programmable Gate Array), for example. Furthermore, some of the functions of the aerial operation detection unit 1350 may be realized by software using a spatial operation detection program executed by the control unit 1110, for example.

[0053] The aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be configured to be built into the space-floating image display device 1000, or may be provided externally as a separate entity from the space-floating image display device 1000. When provided as a separate entity from the space-floating image display device 1000, the aerial operation detection sensor 1351 and the aerial operation detection unit 1350 are configured to be able to transmit information and signals to the space-floating image display device 1000 via a wired or wireless communication connection path or a video signal transmission path.

[0054] Also, the aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be provided separately. This makes it possible to build a system in which the space-floating image display device 1000 without the aerial operation detection function is used as the main body, and only the aerial operation detection function can be added as an option. Also, a configuration in which only the aerial operation detection sensor 1351 is provided separately, and the aerial operation detection unit 1350 is built into the space-floating image display device 1000 may be used. In cases such as when it is desired to more freely position the aerial operation detection sensor 1351 relative to the installation position of the space-floating image display device 1000, a configuration in which only the aerial operation detection sensor 1351 is provided separately is advantageous.

[0055] The imaging unit 1180 is a camera with an image sensor, and captures images of the space near the floating-in-space image 3 and / or the face, arms, fingers, etc. of the user 230. A plurality of imaging units 1180 may be provided. By using a plurality of imaging units 1180, or by using an imaging unit with a depth sensor, the mid-air operation detection unit 1350 can be assisted in detecting the touch operation of the floating-in-space image 3 by the user 230. The imaging unit 1180 may be provided separately from the floating-in-space image display device 1000. When the imaging unit 1180 is provided separately from the floating-in-space image display device 1000, it is sufficient to configure it so that an imaging signal can be transmitted to the floating-in-space image display device 1000 via a wired or wireless communication connection path or the like.

[0056] For example, if the aerial operation detection sensor 1351 is configured as an object intrusion sensor that targets a plane (intrusion detection plane) including the display surface of the spatial floating image 3 and detects whether or not an object has intruded into this intrusion detection plane, the aerial operation detection sensor 1351 may not be able to detect information such as how far an object (e.g., a user's finger) that has not intruded into the intrusion detection plane is from the intrusion detection plane, or how close the object is to the intrusion detection plane.

[0057] In such a case, the distance between the object and the intrusion detection plane can be calculated by using information such as object depth calculation information based on the captured images of the multiple imaging units 1180 and object depth information from the depth sensor. These pieces of information and various pieces of information such as the distance between the object and the intrusion detection plane are used for various display controls for the floating in space image 3.

[0058] Furthermore, without using the mid-air operation detection sensor 1351, the mid-air operation detection unit 1350 may detect a touch operation on the floating-in-space image 3 by the user 230 based on the captured image by the imaging unit 1180.

[0059] Furthermore, the imaging unit 1180 may capture an image of the face of the user 230 operating the space-floating image 3, and the control unit 1110 may perform an identification process for the user 230. Furthermore, in order to determine whether or not there is another person standing around or behind the user 230 operating the space-floating image 3 and peeking at the operation of the user 230 on the space-floating image 3, the imaging unit 1180 may capture an image of a range including the user 230 operating the space-floating image 3 and the surrounding area of ​​the user 230.

[0060] The operation input unit 1107 is, for example, an operation button, a signal receiving unit such as a remote controller, or an infrared light receiving unit, and inputs a signal for an operation different from the air operation (touch operation) by the user 230. Apart from the above-mentioned user 230 who touches the space floating image 3, the operation input unit 1107 may also be used by, for example, an administrator to operate the space floating image display device 1000.

[0061] The video signal input unit 1131 is connected to an external video output device and inputs video data. The video signal input unit 1131 can be any of various digital video input interfaces. For example, HDMI (registered trademark) The video signal input unit 1131 may be configured with a video input interface conforming to the Digital Visual Interface (DVI) standard, a video input interface conforming to the DisplayPort standard, or the like. Alternatively, an analog video input interface such as analog RGB or composite video may be provided. The audio signal input unit 1133 connects to an external audio output device and inputs audio data. The audio signal input unit 1133 may be configured with an audio input interface conforming to the HDMI standard, an optical digital terminal interface, a coaxial digital terminal interface, or the like. In the case of an HDMI standard interface, the video signal input unit 1131 and the audio signal input unit 1133 may be configured as an interface in which a terminal and a cable are integrated. The audio output unit 1140 can output audio based on the audio data input to the audio signal input unit 1133. The audio output unit 1140 may be configured with a speaker. The audio output unit 1140 may also output built-in operation sounds and error warning sounds. Alternatively, the audio output unit 1140 may be configured to output a digital signal to an external device, like the Audio Return Channel function defined in the HDMI standard.

[0062] The nonvolatile memory 1108 stores various data used by the space floating image display device 1000. The data stored in the nonvolatile memory 1108 includes, for example, data for various operations to be displayed on the space floating image 3, display icons, data and layout information for objects to be operated by user operations, etc. The memory 1109 stores image data to be displayed as the space floating image 3, data for controlling the device, etc.

[0063] The control unit 1110 controls the operation of each connected unit. In addition, the control unit 1110 may cooperate with a program stored in the memory 1109 to perform calculations based on information acquired from each unit in the space floating image display device 1000.

[0064] The communication unit 1132 communicates with external devices, external servers, etc. via a wired or wireless communication interface. If the communication unit 1132 has a wired communication interface, the wired communication interface may be configured, for example, as an Ethernet LAN interface. If the communication unit 1132 has a wireless communication interface, the interface may be configured, for example, as a Wi-Fi communication interface, a Bluetooth communication interface, or a mobile communication interface such as 4G or 5G. Various types of data, such as video data, image data, and audio data, are transmitted and received through communication via the communication unit 1132.

[0065] Furthermore, the removable media interface 1134 is an interface for connecting a removable recording medium (removable media). The removable recording medium (removable media) may be composed of a semiconductor device memory such as a solid state drive (SSD), a magnetic recording medium recording device such as a hard disk drive (HDD), or an optical recording medium such as an optical disk. The removable media interface 1134 can read various information such as video data, image data, and audio data recorded on the removable recording medium. The video data, image data, etc. recorded on the removable recording medium are output as the floating image 3 via the video display unit 1102 and the retroreflection unit 1101.

[0066] The storage unit 1170 is a storage device that records various types of information such as video data, image data, audio data, etc. The storage unit 1170 may be configured with a magnetic recording medium recording device such as a hard disk drive (HDD), or a semiconductor element memory such as a solid state drive (SSD). For example, various types of information such as video data, image data, audio data, etc. may be recorded in advance in the storage unit 1170 at the time of product shipment. Furthermore, the storage unit 1170 may record various types of information such as video data, image data, audio data, etc. acquired from an external device, an external server, etc. via the communication unit 1132.

[0067] The video data, image data, etc. recorded in the storage unit 1170 are output as the space floating image 3 via the video display unit 1102 and the retroreflection unit 1101. The video data, image data, etc. of the display icons and objects for the user to operate, etc., displayed as the space floating image 3, are also recorded in the storage unit 1170.

[0068] Layout information of display icons, objects, etc. displayed as the space floating image 3, and various metadata information related to the objects, etc. are also recorded in the storage unit 1170. The audio data recorded in the storage unit 1170 is output as audio from the audio output unit 1140, for example.

[0069] The video control unit 1160 performs various controls related to the video signal input to the video display unit 1102. The video control unit 1160 may be called a video processing circuit, and may be configured with hardware such as an ASIC, FPGA, or video processor. The video control unit 1160 may also be called a video processing unit or an image processing unit. The video control unit 1160 controls video switching, such as which video signal to input to the video display unit 1102, between the video signal to be stored in the memory 1109 and the video signal (video data) input to the video signal input unit 1131, for example.

[0070] In addition, the video control unit 1160 may generate a superimposed video signal by superimposing the video signal to be stored in the memory 1109 and the video signal input from the video signal input unit 1131, and input the superimposed video signal to the video display unit 1102, thereby performing control to form the composite video as the floating-in-space video 3.

[0071] Furthermore, the video control unit 1160 may control image processing of the video signal input from the video signal input unit 1131, the video signal to be stored in the memory 1109, etc. Examples of image processing include scaling processing to enlarge, reduce, deform, etc. the image, brightness adjustment processing to change the brightness, contrast adjustment processing to change the contrast curve of the image, and Retinex processing to decompose the image into light components and change the weighting of each component.

[0072] Furthermore, the video control unit 1160 may perform special effect video processing or the like to assist the aerial operation (touch operation) of the user 230 on the video signal input to the video display unit 1102. The special effect video processing is performed based on, for example, the detection result of the touch operation of the user 230 by the aerial operation detection unit 1350 and the image of the user 230 captured by the imaging unit 1180.

[0073] The attitude sensor 1113 is a sensor configured by a gravity sensor or an acceleration sensor, or a combination of these, and can detect the attitude in which the space-floating image display device 1000 is installed. Based on the attitude detection result of the attitude sensor 1113, the control unit 1110 may control the operation of each connected unit. For example, when an undesirable attitude in the user's usage state is detected, the control unit 1110 may perform control such that the image being displayed on the image display unit 1102 is stopped and an error message is displayed to the user. Alternatively, when the attitude sensor 1113 detects a change in the installation attitude of the space-floating image display device 1000, the control unit 1110 may perform control such that the display direction of the image being displayed on the image display unit 1102 is rotated.

[0074] As explained above, various functions are installed in the space-floating image display device 1000. However, the space-floating image display device 1000 does not need to have all of these functions, and any configuration is acceptable as long as it has the function of forming the space-floating image 3.

[0075] <Configuration example of a space floating image display device> Next, a configuration example of the space-floating image display device will be explained. The layout of the components of the space-floating image display device according to this embodiment can be various depending on the usage form. Below, the layouts of each of Figs. 4A to 4M will be explained. In addition, in each example of Figs. 4A to 4M, the thick line surrounding the space-floating image display device 1000 indicates an example of the housing structure of the space-floating image display device 1000.

[0076] FIG. 4A is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4A is equipped with an optical system corresponding to the optical system of FIG. 2A. The space-floating image display device 1000 shown in FIG. 4A is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4A, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. When the mid-air operation detection sensor 1351 is installed as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230. Note that the x direction is the left-right direction as seen from the user, the y direction is the front-back direction (depth direction) as seen from the user, and the z direction is the up-down direction (vertical direction). Hereinafter, the definitions of the x direction, y direction, and z direction are the same in each drawing of FIG. 4, so repeated explanations will be omitted.

[0077] FIG. 4B is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4B is equipped with an optical system corresponding to the optical system of FIG. 2A. The space-floating image display device 1000 shown in FIG. 4B is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (toward the user 230). That is, in FIG. 4B, the space-floating image display device is installed with the transparent member 100 facing the front of the device (toward the user 230). The space-floating image 3 is formed on the user 230 side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230. 4B, the mid-air operation detection sensor 1351 can use the reflection of sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, the reflectivity of the nail is higher than that of the finger pad, so this configuration can improve the accuracy of touch detection.

[0078] FIG. 4C is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4C is equipped with an optical system corresponding to the optical system of FIG. 2B. The space-floating image display device 1000 shown in FIG. 4C is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4C, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the user's 230 finger.

[0079] FIG. 4D is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4D is equipped with an optical system corresponding to the optical system of FIG. 2B. The space-floating image display device 1000 shown in FIG. 4D is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (toward the user 230). That is, in FIG. 4D, the space-floating image display device 1000 is installed with the transparent member 100 facing the front of the device (toward the user 230). The space-floating image 3 is formed on the user 230 side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the user 230's finger. 4D, the mid-air operation detection sensor 1351 can use the reflection of sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, the reflectivity of the nail is higher than that of the finger pad, so this configuration can improve the accuracy of touch detection.

[0080] FIG. 4E is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4E is equipped with an optical system corresponding to the optical system of FIG. 2C. The space-floating image display device 1000 shown in FIG. 4E is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4E, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels directly upward. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230.

[0081] FIG. 4F is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4F is equipped with an optical system corresponding to the optical system of FIG. 2C. The space-floating image display device 1000 shown in FIG. 4F is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (toward the user 230). That is, in FIG. 4F, the space-floating image display device 1000 is installed with the transparent member 100 facing the front of the device (toward the user 230). The space-floating image 3 is formed on the user 230 side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels toward the user. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the user 230's finger.

[0082] FIG. 4G is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4G is equipped with an optical system corresponding to the optical system shown in FIG. 2C. In the optical systems of the space-floating image display devices shown in FIGS. 4A to 4F, the central optical path of the image light emitted from the display device 1 was on the yz plane. That is, in the optical systems of the space-floating image display devices shown in FIGS. 4A to 4F, the image light traveled in the front-to-back and up-to-down directions as seen from the user. In contrast, in the optical system of the space-floating image display device shown in FIG. 4G, the central optical path of the image light emitted from the display device 1 is on the xy plane. That is, in the optical system of the space-floating image display device shown in FIG. 4G, the image light travels in the left-to-right and front-to-back directions as seen from the user. The space-floating image display device 1000 shown in FIG. 4G is installed so that the surface on which the space-floating image 3 is formed faces the front of the device (toward the user 230). That is, in Fig. 4G, the space-floating image display device 1000 has the transparent member 100 installed on the front side of the device (toward the user 230). The space-floating image 3 is formed on the user side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels towards the user. If the mid-air operation detection sensor 1351 is installed as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230.

[0083] FIG. 4H is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4H differs from the space-floating image display device of FIG. 4G in that it has a window with a transparent plate 100B made of glass or plastic on the back of the device (opposite the position where the user 230 views the space-floating image 3, i.e., opposite the traveling direction of the image light of the space-floating image 3 toward the user 230). The rest of the configuration is the same as that of the space-floating image display device of FIG. 4G, so repeated explanations will be omitted. The space-floating image display device 1000 of FIG. 4H has a window with a transparent plate 100B on the opposite side of the traveling direction of the image light of the space-floating image 3 from the space-floating image 3. Therefore, when the user 230 views the space-floating image 3, they can recognize the scenery behind the space-floating image display device 1000 as the background of the space-floating image 3. Therefore, the user 230 can perceive the space floating image 3 as floating in the air in front of the scenery behind the space floating image display device 1000. This can further emphasize the feeling of the space floating image 3 floating in the air.

[0084] Depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separator 101B, a portion of the image light output from the display device 1 may be reflected by the polarization separator 101B and head toward the transparent plate 100B. Depending on the coating performance of the surface of the transparent plate 100B, this light may be reflected again by the surface of the transparent plate 100B and be visible to the user as stray light. Therefore, in order to prevent this stray light, the transparent plate 100B may not be provided in the window on the back of the space-floating image display device 1000.

[0085] FIG. 4I is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4I differs from the space-floating image display device of FIG. 4H in that a light-blocking door 1410 is provided at the window of a transparent plate 100B located on the back of the device (the opposite side from where the user 230 views the space-floating image 3). Other configurations are the same as those of the space-floating image display device of FIG. 4H , so repeated explanations will be omitted. The door 1410 of the space-floating image display device 1000 of FIG. 4I has, for example, a light-blocking plate and a mechanism for moving (sliding), rotating, or attaching / detaching the light-blocking plate, thereby switching between an open state and a light-blocking state for the window of the transparent plate 100B located at the back of the space-floating image display device 1000 (the rear window). The movement (sliding) and rotation of the light-blocking plate by the door 1410 may be electrically driven by a motor (not shown). The motor may be controlled by the control unit 1110 of FIG. 3. 4I shows an example in which the number of light-shielding plates in the opening and closing door 1410 is two. However, the number of light-shielding plates in the opening and closing door 1410 may be one.

[0086] For example, when the view seen through the window of the transparent plate 100B of the space-floating image display device 1000 is outdoors, the brightness of sunlight varies depending on the weather. When the outdoor sunlight is strong, the background of the space-floating image 3 may become too bright, reducing the user 230's visibility of the space-floating image 3. In such a case, by moving (sliding), rotating, or attaching the light blocking plate of the opening / closing door 1410 to block the light from the rear window, the background of the space-floating image 3 becomes dark, thereby relatively increasing the visibility of the space-floating image 3. Such a blocking operation by the light blocking plate of the opening / closing door 1410 may be performed directly by the force of the user 230's hand. In response to an operation input via the operation input unit 1107 of FIG. 3, the control unit 1110 may control a motor (not shown) to perform the blocking operation by the light blocking plate of the opening / closing door 1410.

[0087] An illuminance sensor may be provided on the rear side (opposite the user 230) of the space-floating image display device 1000, such as near the rear window, to measure the brightness of the space beyond the rear window. In this case, the control unit 1110 of Fig. 3 may control a motor (not shown) to perform the opening and closing operation of the light blocking plate of the opening and closing door 1410 according to the detection result of the illuminance sensor. By controlling the opening and closing operation of the light blocking plate of the opening and closing door 1410 in this way, it becomes possible to more suitably maintain the visibility of the space-floating image 3, even if the user 230 does not manually open and close the light blocking plate of the opening and closing door 1410.

[0088] Furthermore, the light blocking plate by the opening and closing door 1410 may be manually detachable. Depending on the intended use and installation environment of the space floating image display device 1000, the user can select whether to leave the rear window open or in a light blocking state. If it is planned to use the rear window in a light blocking state for a long period of time, the detachable light blocking plate can be fixed in the light blocking state. Also, if it is planned to use the rear window in an open state for a long period of time, it can be used with the detachable light blocking plate removed. The light blocking plate may be attached and detached using screws, a hook structure, or a fitting structure.

[0089] Even in the example of the space-floating image display device 1000 shown in FIG. 4I, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separator 101B, a portion of the image light output from the display device 1 may be reflected by the polarization separator 101B and directed toward the transparent plate 100B. Depending on the coating performance of the surface of the transparent plate 100B, this light may be reflected again by the surface of the transparent plate 100B and be perceived by the user as stray light. Therefore, to prevent this stray light, the window on the back of the space-floating image display device 1000 may be configured without the transparent plate 100B. The above-described opening / closing door 1410 may be provided in a window that does not have the transparent plate 100B. To prevent this stray light, it is desirable that the inner surface of the housing of the light-shielding plate of the above-described opening / closing door 1410 have a coating or material with low light reflectance.

[0090] FIG. 4J is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4J differs from the space-floating image display device of FIG. 4H in that instead of the transparent plate 100B made of glass or plastic, an electronically controlled transmittance variable device 1620 is provided on the rear window. The other components are the same as those of the space-floating image display device of FIG. 4H, and therefore will not be described again. An example of the electronically controlled transmittance variable device 1620 is a liquid crystal shutter. That is, a liquid crystal shutter can control the transmitted light by controlling the voltage of a liquid crystal element sandwiched between two polarizing plates. Therefore, by controlling the liquid crystal shutter to increase the transmittance, the background of the space-floating image 3 can be seen through the scenery through the rear window. Furthermore, by controlling the liquid crystal shutter to increase the transmittance, the scenery through the rear window can be hidden as the background of the space-floating image 3. Furthermore, since the liquid crystal shutter can be controlled to an intermediate length, it can also be set to a transmittance of 50%, for example. For example, the control unit 1110 may control the transmittance of the electronically controlled transmittance varying device 1620 in response to an operation input via the operation input unit 1107 in Fig. 3. With this configuration, in cases where a viewer wants to see the scenery through the rear window as the background of the Space Floating Image 3, but the scenery through the rear window as the background is too bright and reduces the visibility of the Space Floating Image 3, the visibility of the Space Floating Image 3 can be adjusted by adjusting the transmittance of the electronically controlled transmittance varying device 1620.

[0091] In addition, an illuminance sensor may be provided on the back side (opposite the user 230) of the space-floating image display device 1000, such as near the rear window, to measure the brightness of the space beyond the rear window. In this case, the control unit 1110 in Fig. 3 controls the transmittance of the electronically controlled transmittance variable device 1620 according to the detection result of the illuminance sensor. In this way, even if the user 230 does not perform an operation input via the operation input unit 1107 in Fig. 3, the transmittance of the electronically controlled transmittance variable device 1620 can be adjusted according to the brightness of the space beyond the rear window, making it possible to more suitably maintain the visibility of the space-floating image 3.

[0092] In the above example, a liquid crystal shutter has been described as an example of the electronically controlled variable transmittance device 1620. However, electronic paper may be used as another example of the electronically controlled variable transmittance device 1620. The same effects as those described above can be obtained when electronic paper is used. Furthermore, electronic paper consumes very little power to maintain a halftone state. Therefore, a space floating image display device with lower power consumption can be realized compared to when a liquid crystal shutter is used.

[0093] Fig. 4K is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4K differs from the space-floating image display device of Fig. 4G in that it has a transmissive self-luminous image display device 1650 instead of the transparent member 100. The other configurations are the same as those of the space-floating image display device of Fig. 4G, so repeated explanations will be omitted.

[0094] In the space-floating image display device 1000 of FIG. 4K, a light beam of an image passes through the display surface of the transmissive self-luminous image display device 1650, and then a space-floating image 3 is formed outside the space-floating image display device 1000. That is, when an image is displayed on the transmissive self-luminous image display device 1650, which is a two-dimensional flat display, the space-floating image 3 can be displayed as a pop-up image further in front of the image displayed on the transmissive self-luminous image display device 1650. In this case, the user 230 can simultaneously view two images at different depth positions. The transmissive self-luminous image display device 1650 may be configured using existing technology such as a transmissive organic EL panel disclosed in, for example, Japanese Patent Application Laid-Open No. 2014-216761. Although not shown in FIG. 3, the transmissive self-luminous image display device 1650 may be configured as a component of the space-floating image display device 1000 of FIG. 3 and connected to other processing units such as the control unit 1110.

[0095] Here, if the transmissive self-luminous video display device 1650 displays both the background and an object such as a character, and then the object such as the character moves to the foreground, the floating video image 3, a more effective surprise video experience can be provided to the user 230.

[0096] Furthermore, if the inside of the space-floating image display device 1000 is kept in a light-blocking state, the background of the transmissive self-luminous image display device 1650 becomes sufficiently dark. Therefore, when no image is displayed on the display device 1 or the light source of the display device 1 is turned off and an image is displayed only on the transmissive self-luminous image display device 1650, the transmissive self-luminous image display device 1650 appears to the user 230 as a normal two-dimensional flat display rather than a transmissive display (since the space-floating image 3 in the embodiment of the present invention is displayed as a real optical image in a space without a screen, if the light source of the display device 1 is turned off, the intended display position of the space-floating image 3 becomes empty space). Therefore, when the transmissive self-luminous image display device 1650 is used to display an image as if it were a normal two-dimensional flat display, characters, objects, etc. can be suddenly displayed in the air as the space-floating image 3, thereby providing the user 230 with a more effective surprise video experience.

[0097] Note that the darker the interior of the space-floating image display device 1000, the more the transmissive self-luminous image display device 1650 appears like a two-dimensional flat display. Therefore, an absorptive polarizer (not shown) that transmits the polarized waves of the image light reflected by the polarization separation member 101B and absorbs polarized waves that are 90° out of phase with the polarized waves may be provided on the surface of the transmissive self-luminous image display device 1650 facing the interior of the space-floating image display device 1000 (the surface where the image light reflected by the polarization separation member 101B enters the transmissive self-luminous image display device 1650, i.e., the surface of the transmissive self-luminous image display device 1650 opposite the space-floating image 3). This does not have a significant effect on the image light that forms the space-floating image 3, but it can significantly reduce the light that enters the interior of the space-floating image display device 1000 from the outside through the transmissive self-luminous image display device 1650, making the interior of the space-floating image display device 1000 darker, which is preferable.

[0098] 4L is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4L is a modified example of the space-floating image display device of FIG. 4K. The orientation of the components in the space-floating image display device 1000 is different from that of the space-floating image display device of FIG. 4K, and is closer to the arrangement of the space-floating image display device of FIG. 4F. The functions and operations of each component are the same as those of the space-floating image display device of FIG. 4K, so repeated explanations will be omitted.

[0099] In the space-floating image display device of FIG. 4L, after the luminous flux of image light passes through the transmissive self-luminous image display device 1650, a space-floating image 3 is formed on the user 230 side of the transmissive self-luminous image display device 1650.

[0100] In both the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L, the space-floating image 3 is displayed superimposed on the image of the transmissive self-luminous image display device 1650 as seen by the user 230. Here, the position of the space-floating image 3 and the position of the image of the transmissive self-luminous image display device 1650 are configured to have a difference in the depth direction. Therefore, when the user moves their head (the position of the viewpoint), they can recognize the depth of the two images due to parallax. Therefore, by displaying two images at different depth positions, it is possible to provide the user with a more suitable three-dimensional image experience with the naked eye without the need for stereoscopic glasses or the like.

[0101] Fig. 4M is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4M is provided with a second display device 1680 on the rear side as seen from the user relative to the polarization separation member 101B of the space-floating image display device of Fig. 4G. The other configurations are the same as those of the space-floating image display device of Fig. 4G, so repeated explanations will be omitted.

[0102] In the configuration example shown in FIG. 4M, the second display device 1680 is provided behind the display position of the space-floating image 3, and its image display surface faces the space-floating image 3. With this configuration, from the user 230's perspective, the image of the second display device 1680 and the space-floating image 3, which are displayed at two different depth positions, can be viewed superimposed on each other. In other words, the second display device 1680 is positioned so as to display an image in the direction of the user 230 viewing the space-floating image 3. Although the second display device 1680 is not shown in FIG. 3, it may be configured to be connected to other processing units such as the control unit 1110 as one component of the space-floating image display device 1000 of FIG. 3.

[0103] Note that the image light of the second display device 1680 of the space-floating image display device 1000 of FIG. 4M is viewed by the user 230 after passing through the polarization separator 101B. Therefore, in order for the image light of the second display device 1680 to more suitably pass through the polarization separator 101B, it is desirable that the image light output from the second display device 1680 be polarized in a vibration direction that the polarization separator 101B more suitably transmits. That is, it is desirable that the image light be polarized in the same vibration direction as the polarization of the image light output from the display device 1. For example, if the image light output from the display device 1 is S-polarized, it is desirable that the image light output from the second display device 1680 is also S-polarized. Furthermore, if the image light output from the display device 1 is P-polarized, it is desirable that the image light output from the second display device 1680 is also P-polarized.

[0104] The example of the space-floating image display device of FIG. 4M also has the same effect as the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L in that a second image is displayed behind the space-floating image 3. However, unlike the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L, in the example of the space-floating image display device of FIG. 4M, the luminous flux of image light for forming the space-floating image 3 does not pass through the second display device 1680. Therefore, the second display device 1680 does not need to be a transmissive self-luminous image display device, but may be a liquid crystal display, which is a two-dimensional flat display. The second display device 1680 may also be an organic EL display. Therefore, the example of the space-floating image display device of FIG. 4M can realize the space-floating image display device 1000 at a lower cost than the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L.

[0105] Here, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, a portion of the image light output from the display device 1 may be reflected by the polarization separation member 101B and travel toward the second display device 1680. This light (a portion of the image light) may be reflected again by the surface of the second display device 1680 and may be visually recognized by the user as stray light.

[0106] Therefore, to prevent this stray light, an absorptive polarizer may be provided on the surface of the second display device 1680. In this case, the absorptive polarizer may be an absorptive polarizer that transmits the polarized waves of the image light output from the second display device 1680 and absorbs polarized waves that are 90° out of phase with the polarized waves of the image light output from the second display device 1680. If the second display device 1680 is a liquid crystal display, an absorptive polarizer is also provided on the image output side of the liquid crystal display. However, if there is a cover glass (cover glass on the image display surface side) on the output surface of the absorptive polarizer on the image output side of the liquid crystal display, it is not possible to prevent stray light caused by reflection of the cover glass by light from outside the liquid crystal display. Therefore, it is necessary to separately provide the above-mentioned absorptive polarizer on the surface of the cover glass.

[0107] When an image is displayed on the second display device 1680, which is a two-dimensional flat display, the floating-in-space image 3 can be displayed as an image further in front of the image on the second display device 1680. In this case, the user 230 can simultaneously view two images at different depth positions. By displaying a character on the floating-in-space image 3 and a background on the second display device 1680, it is possible to provide the effect that the user 230 is viewing the space in which the character exists in a three-dimensional manner.

[0108] Furthermore, if the second display device 1680 displays both the background and an object such as a character, and then the object such as the character moves to the foreground in the floating image 3, it is possible to provide the user 230 with a more effective surprise visual experience.

[0109] <Display device> Next, the display device 1 of this embodiment will be described with reference to the drawings. The display device 1 of this embodiment includes an image display element 11 (liquid crystal display panel) and a light source device 13 that constitutes its light source. Fig. 5 shows the light source device 13 together with the liquid crystal display panel as an exploded perspective view.

[0110] As shown by arrow 30 in Figure 5, this liquid crystal display panel (image display element 11) receives an illumination light beam from light source device 13, which is a backlight device, that has narrow-angle diffusion characteristics, i.e., has strong directivity (straightness) and characteristics similar to laser light with a polarization plane aligned in one direction. The liquid crystal display panel (image display element 11) modulates the received illumination light beam in accordance with an input video signal. The modulated image light is reflected by retroreflector 2 and passes through transparent member 100 to form a real image, a floating image in space (see Figure 1).

[0111] 5 also shows a configuration including a liquid crystal display panel 11 constituting the display device 1, a light redirection panel 54 that controls the directional characteristics of the light beam emitted from the light source device 13, and a narrow-angle diffuser (not shown) as needed. Specifically, polarizing plates are provided on both sides of the liquid crystal display panel 11, and image light of a specific polarization is emitted with its intensity modulated by a video signal (see arrow 30 in FIG. 5). This allows a desired image to be projected as highly directional (linearly propagating) light of a specific polarization via the light redirection panel 54 toward the retroreflector 2. After being reflected by the retroreflector 2, the light is transmitted toward the eyes of an observer outside the store (space), forming a floating image 3. A protective cover 50 (see FIGS. 6 and 7) may be provided on the surface of the light redirection panel 54.

[0112] <Display device example 1> FIG. 6 shows an example of a specific configuration of the display device 1. In FIG. 6, a liquid crystal display panel 11 and a light direction conversion panel 54 are disposed on the light source device 13 shown in FIG. 5. The light source device 13 is configured on a case shown in FIG. 5, which is formed of, for example, plastic and contains LED elements 201 and a light guide 203. As shown in FIG. 5 and other figures, the end surface of the light guide 203 is provided with a lens shape whose cross-sectional area gradually increases toward the light receiving section in order to convert the divergent light from each LED element 201 into a substantially parallel beam. The lens shape has an effect of gradually reducing the divergence angle by multiple total reflections during propagation within the light guide 203. The liquid crystal display panel 11 constituting the display device 1 is attached to the top surface of the display device 1. In addition, LED (Light Emitting Diode) elements 201, which are semiconductor light sources, and an LED board 202 on which their control circuits are mounted are attached to one side surface (the left end surface in this example) of the case of the light source device 13. A heat sink, which is a member for cooling the heat generated by the LED elements and the control circuit, may be attached to the outer surface of the LED substrate 202.

[0113] The liquid crystal display panel frame (not shown) is attached to the top surface of the case of the light source device 13. The liquid crystal display panel 11 is attached to the frame, and an FPC (Flexible Printed Circuits) (not shown) electrically connected to the liquid crystal display panel 11 is also attached to the frame. That is, the liquid crystal display panel 11, which is the image display element, generates a display image by modulating the intensity of transmitted light in conjunction with the LED elements 201, which are solid-state light sources, based on a control signal from a control circuit (image control unit 1160 in FIG. 3) constituting the electronic device. The generated image light has a narrow diffusion angle and contains only specific polarization components, resulting in a novel image display device similar to a surface-emitting laser image source driven by a video signal. Currently, it is technically and safety-wise impossible to obtain a laser beam of the same size as the image obtained by the display device 1 described above using a laser device. Therefore, in this embodiment, light similar to the surface-emitting laser image light described above is obtained from a beam of light from a general light source, for example, an LED element.

[0114] Next, the configuration of the optical system housed in the case of the light source device 13 will be described in detail with reference to FIG. 7 as well as FIG.

[0115] 6 and 7 are cross-sectional views, and only one of the multiple LED elements 201 constituting the light source is shown, and this light is converted into approximately collimated light by the shape of the light-receiving end surface 203a of the light guide 203. For this reason, the light-receiving portion of the light guide end surface and the LED element are attached while maintaining a predetermined positional relationship.

[0116] Each light guide 203 is formed of a translucent resin such as acrylic. The LED light receiving surface at the end of light guide 203 has a cone-shaped outer periphery obtained by rotating a parabolic cross section, and at the top of the light guide 203, a concave portion with a convex portion (i.e., a convex lens surface) formed in the center is formed, and at the center of the flat portion, a convex lens surface (or a concave lens surface) that protrudes outward (not shown). The outer shape of the light receiving portion of the light guide to which LED element 201 is attached is a parabolic shape that forms a cone-shaped outer periphery, and is set within an angle range that allows total reflection within the light that is emitted from the LED element toward the periphery, or a reflective surface is formed.

[0117] On the other hand, the LED elements 201 are arranged at predetermined positions on the surface of the circuit board, that is, the LED substrate 202. The LED substrate 202 is arranged and fixed to the LED collimator (light-receiving end surface 203a) so that the LED elements 201 on the surface are positioned in the center of the recessed portion described above.

[0118] According to this configuration, the shape of the light-receiving end surface 203a of the light guide 203 makes it possible to extract the light emitted from the LED element 201 as approximately parallel light, thereby improving the efficiency of use of the generated light.

[0119] As described above, the light source device 13 is configured by attaching a light source unit in which a plurality of LED elements 201 serving as light sources are arranged to the light-receiving end surface 203a, which is a light-receiving section provided on the end surface of the light guide 203, and the divergent light beams from the LED elements 201 are converted into approximately parallel light by the lens shape of the light-receiving end surface 203a of the light guide end surface, which is then guided inside the light guide 203 (in a direction parallel to the drawing) as shown by the arrow, and emitted by the light beam direction conversion means 204 toward the liquid crystal display panel 11, which is disposed approximately parallel to the light guide 203 (in a direction perpendicular to the front of the drawing). The uniformity of the light beam incident on the liquid crystal display panel 11 can be controlled by optimizing the distribution (density) of the light beam direction conversion means 204 depending on the shape inside or on the surface of the light guide.

[0120] The light beam direction conversion means 204 described above emits the light beam propagated inside the light guide toward the liquid crystal display panel 11 (in a direction perpendicular to the front of the drawing) which is disposed substantially parallel to the light guide 203, by changing the shape of the surface of the light guide or by providing a portion with a different refractive index inside the light guide. In this case, when the liquid crystal display panel 11 is faced directly at the center of the screen and the viewpoint is positioned at the same position as the diagonal dimension of the screen, if the relative brightness ratio between the center of the screen and the periphery of the screen is 20% or more, there is no practical problem, and if it exceeds 30%, it will be an even better characteristic.

[0121] 6 is a cross-sectional layout diagram illustrating the configuration and operation of a light source of this embodiment that performs polarization conversion in light source device 13 including light guide 203 and LED element 201. In Fig. 6, light source device 13 is composed of light guide 203 formed of, for example, plastic or the like and having light beam direction conversion means 204 on its surface or inside, LED element 201 as a light source, reflective sheet 205, retardation plate 206, lenticular lens, etc., and on the upper surface of light source device 13 is attached liquid crystal display panel 11 that has polarizing plates on the light source light entrance surface and the image light exit surface.

[0122] In addition, a film- or sheet-like reflective polarizing plate 49 is provided on the light source light incident surface (bottom surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, selectively reflecting one polarization (e.g., P-wave) 212 of the natural light beam 210 emitted from the LED element 201. The reflected light is reflected again by a reflective sheet 205 provided on one surface (bottom surface in the figure) of the light guide 203 and directed toward the liquid crystal display panel 11. Therefore, a retardation plate (λ / 4 plate) is provided between the reflective sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49. The reflected light beam is reflected by the reflective sheet 205 and passes through it twice, converting the reflected light beam from P-polarized to S-polarized, thereby improving the utilization efficiency of the light source light as image light. The image light beam, the light intensity of which is modulated by a video signal in the liquid crystal display panel 11 (arrow 213 in Figure 6), enters the retroreflector 2. After reflection by the retroreflector 2, a real, floating image can be obtained.

[0123] 7 is a cross-sectional layout diagram illustrating the configuration and operation of a light source of this embodiment that performs polarization conversion in light source device 13 including light guide 203 and LED elements 201, similar to Fig. 6. Light source device 13 is similarly composed of light guide 203 formed of, for example, plastic and having light beam direction conversion means 204 on its surface or inside, LED elements 201 as a light source, reflective sheet 205, retardation plate 206, lenticular lens, etc. On the top surface of light source device 13, a liquid crystal display panel 11 is attached as an image display element, which has polarizing plates on the light source light entrance surface and the image light exit surface.

[0124] A film or sheet-like reflective polarizing plate 49 is provided on the light source light incident surface (bottom surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, selectively reflecting one polarized wave (e.g., S wave) 211 of the natural light beam 210 emitted from the LED element 201. In other words, the selective reflection characteristics of the reflective polarizing plate 49 in the example of FIG. 7 differ from those in FIG. 7. The reflected light is reflected by a reflective sheet 205 provided on one surface (bottom surface in the figure) of the light guide 203 and returns to the liquid crystal display panel 11. A retardation plate (λ / 4 plate) is provided between the reflective sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49, and the reflected light beam is reflected by the reflective sheet 205 and passes through it twice, converting it from S-polarized light to P-polarized light, thereby improving the utilization efficiency of the light source light as image light. The image light beam intensity-modulated by the image signal in the liquid crystal display panel 11 (arrow 214 in FIG. 7) enters the retroreflector 2. After reflection by the retroreflector 2, a real image, a floating image in space, can be obtained.

[0125] In the light source devices shown in Figures 6 and 7, in addition to the function of the polarizer provided on the light incident surface of the corresponding liquid crystal display panel 11, the reflective polarizer reflects the polarized light component on one side, so the theoretically obtainable contrast ratio is the reciprocal of the cross transmittance of the reflective polarizer multiplied by the reciprocal of the cross transmittance obtained by the two polarizers attached to the liquid crystal display panel. This results in high contrast performance. In fact, experiments have confirmed that the contrast performance of the displayed image is improved by more than 10 times. As a result, high-quality images comparable to those of self-luminous organic EL displays are obtained.

[0126] <Display device example 2> FIG. 8 shows another example of the specific configuration of the display device 1. This light source device 13 is configured by housing LEDs, a collimator, a composite diffusion block, a light guide, etc. in a case made of, for example, plastic, and has a liquid crystal display panel 11 attached to its upper surface. Also, on one side of the case of the light source device 13, an LED (Light Emitting Diode) which is a semiconductor light source is mounted. The LED substrate on which the emitting diode (LED) elements 14a and 14b and their control circuits are mounted is In addition, a heat sink 103, which is a member for cooling the heat generated by the LED elements and the control circuit, is attached to the outer surface of the LED substrate.

[0127] The liquid crystal display panel frame attached to the top surface of the case is configured to have attached thereto a liquid crystal display panel 11 attached to the frame, and further to have attached thereto an FPC (Flexible Printed Circuits) 403 electrically connected to the liquid crystal display panel 11. That is, the liquid crystal display panel 11, which is a liquid crystal display element, generates a display image by modulating the intensity of transmitted light together with the LED elements 14a and 14b, which are solid-state light sources, based on a control signal from a control circuit (not shown here) that constitutes the electronic device.

[0128] <Display device example 3> Next, another example of the specific configuration of the display device 1 (Example 3 of the display device) will be described with reference to Fig. 9. The light source device of this display device 1 converts a divergent beam of light (a mixture of P-polarized and S-polarized light) from an LED into a substantially parallel beam by a collimator 18, and reflects the parallel beam toward the liquid crystal display panel 11 by the reflecting surface of a reflective light guide 304. The reflected light is incident on a reflective polarizer 49 disposed between the liquid crystal display panel 11 and the reflective light guide 304. The reflective polarizer 49 transmits light of a specific polarization (e.g., P-polarized light) and causes the transmitted polarized light to be incident on the liquid crystal display panel 11. Here, light of polarization other than the specific polarization (e.g., S-polarized light) is reflected by the reflective polarizer 49 and directed toward the reflective light guide 304 again.

[0129] The reflective polarizing plate 49 is installed at an angle with respect to the liquid crystal display panel 11 so that the reflective polarizing plate 49 is not perpendicular to the chief ray of light from the reflective surface of the reflective light guide 304. The chief ray of the light reflected by the reflective polarizing plate 49 is incident on the transmission surface of the reflective light guide 304. The light that has entered the transmission surface of the reflective light guide 304 passes through the back surface of the reflective light guide 304, passes through the λ / 4 plate 270 which is a retardation plate, and is reflected by the reflector 271. The light reflected by the reflector 271 passes through the λ / 4 plate 270 again, and passes through the transmission surface of the reflective light guide 304. The light that has passed through the transmission surface of the reflective light guide 304 is incident on the reflective polarizing plate 49 again.

[0130] At this time, the light that re-enters the reflective polarizer 49 has passed through the λ / 4 plate 270 twice, and therefore its polarization has been converted to a polarization (for example, P-polarized light) that is transmitted through the reflective polarizer 49. Therefore, the light whose polarization has been converted passes through the reflective polarizer 49 and enters the liquid crystal display panel 11. Note that with regard to the polarization design related to the polarization conversion, the polarization may be configured in reverse from the above explanation (S-polarized light and P-polarized light may be reversed).

[0131] As a result, the light from the LED is aligned to a specific polarization (for example, P polarization), enters the liquid crystal display panel 11, and is brightness-modulated in accordance with the video signal to display an image on the panel surface. As in the above example, multiple LEDs that make up the light source are shown (however, since this is a vertical cross section, only one is shown in Figure 9), and these are attached at predetermined positions relative to the collimator 18.

[0132] Each of the collimators 18 is formed of, for example, a translucent resin such as acrylic or glass. The collimator 18 may have a cone-shaped outer peripheral surface obtained by rotating a parabolic cross section. The collimator 18 may have a concave portion with a convex portion (i.e., a convex lens surface) formed in the center of the apex (the side facing the LED substrate 102). The collimator 18 may have a convex lens surface (or a concave lens surface) protruding outward in the center of the flat portion (the side opposite the apex). The parabolic surface forming the cone-shaped outer peripheral surface of the collimator 18 is set within an angle range that allows total reflection of the light emitted from the LED toward the periphery within the parabolic surface, or a reflective surface is formed therein.

[0133] The LEDs are arranged at predetermined positions on the surface of the circuit board, that is, the LED substrate 102. The LED substrate 102 is arranged and fixed to the collimator 18 so that the LEDs on the surface are positioned at the center of the apex of the convex cone shape (or in the recess if there is a recess at the apex).

[0134] With this configuration, the collimator 18 focuses the light emitted from the LED, particularly the light emitted from the central portion, into parallel light by the convex lens surface that forms the outer shape of the collimator 18. Light emitted from other portions toward the periphery is reflected by the parabolic surface that forms the outer peripheral surface of the conical shape of the collimator 18, and is similarly focused into parallel light. In other words, the collimator 18, which has a convex lens in its center and a parabolic surface formed on its periphery, makes it possible to extract almost all of the light generated by the LED as parallel light, thereby improving the utilization efficiency of the generated light.

[0135] Furthermore, the light converted into approximately parallel light by the collimator 18 shown in FIG. 9 is reflected by the reflective light guide 304. Due to the action of the reflective polarizer 49, light of a specific polarization of the light is transmitted through the reflective polarizer 49, while light of the other polarization reflected by the reflective polarizer 49 is transmitted again through the light guide 304. The light is reflected by the reflector 271, which is located opposite the liquid crystal display panel 11 with respect to the reflective light guide 304. At this time, the light is polarized and converted twice by passing through the λ / 4 plate 270, which is a retardation plate. The light reflected by the reflector 271 is transmitted again through the light guide 304 and enters the reflective polarizer 49 provided on the opposite surface. Since the incident light has been polarization-converted, it passes through the reflective polarizer 49, aligns its polarization direction, and enters the liquid crystal display panel 11. As a result, all of the light from the light source can be utilized, thereby doubling the geometrical optical utilization efficiency of light. Furthermore, since the degree of polarization (extinction ratio) of the reflective polarizer is also included in the extinction ratio of the entire system, the use of the light source device of this embodiment significantly improves the contrast ratio of the entire display device. Adjusting the surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflector 271 allows adjustment of the angle of light reflection and diffusion at each reflective surface. The surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflector 271 can be adjusted for each design to optimize the uniformity of the light incident on the liquid crystal display panel 11.

[0136] It should be noted that the λ / 4 plate 270, which is the retardation plate in Fig. 9, does not necessarily have to have a phase difference of λ / 4 with respect to polarized light that is perpendicularly incident on the λ / 4 plate 270. In the configuration of Fig. 9, any retardation plate may be used as long as the phase changes by 90° (λ / 2) when polarized light passes through it twice. The thickness of the retardation plate may be adjusted according to the incident angle distribution of the polarized light.

[0137] <Display device example 4> Furthermore, another example (Example 4 of Display Device) of the configuration of an optical system such as a light source device of a display device will be described with reference to Fig. 10. This is an example of a configuration in which a diffusion sheet is used instead of reflective light guide 304 in the light source device of Example 3 of the display device. Specifically, two optical sheets (optical sheet 207A and optical sheet 207B) that convert the diffusion characteristics in the vertical and horizontal directions of the drawing (front-to-back directions not shown in the drawing) are used on the light emission side of collimator 18, and light from collimator 18 is made to enter between the two optical sheets (diffusion sheets).

[0138] The optical sheet may be a single sheet instead of a two-sheet configuration. In the case of a single sheet configuration, the vertical and horizontal diffusion characteristics are adjusted by the fine shapes on the front and back surfaces of the single optical sheet. Alternatively, multiple diffusion sheets may be used to share the functions. In the example of FIG. 10, the reflection and diffusion characteristics due to the front and back shapes of optical sheets 207A and 207B can be optimally designed using the number of LEDs, the divergence angle from LED substrate (optical element) 102, and the optical specifications of collimator 18 as design parameters so that the surface density of the light beam emitted from liquid crystal display panel 11 is uniform. In other words, the diffusion characteristics are adjusted by the surface shapes of multiple diffusion sheets instead of light guides.

[0139] In the example of FIG. 10, polarization conversion is performed in the same manner as in the display device example 3 described above. That is, in the example of FIG. 10, reflective polarizing plate 49 may be configured to have the property of reflecting S-polarized light (transmitting P-polarized light). In this case, the reflective polarizing plate 49 transmits P-polarized light out of the light emitted from the LED light source, and the transmitted light enters liquid crystal display panel 11. The reflective polarizing plate 49 reflects S-polarized light out of the light emitted from the LED light source, and the reflected light passes through retardation plate 270 shown in FIG. 10. The light that passes through retardation plate 270 is reflected by reflector 271. The light reflected by reflector 271 passes through retardation plate 270 again and is converted to P-polarized light. The polarization-converted light passes through reflective polarizing plate 49 and enters liquid crystal display panel 11.

[0140] It should be noted that the λ / 4 plate 270, which is the retarder in FIG. 10, does not necessarily have to have a phase difference of λ / 4 with respect to polarized light that is perpendicularly incident on the λ / 4 plate 270. In the configuration of FIG. 10, any retarder that changes the phase by 90° (λ / 2) when polarized light passes through it twice will suffice. The thickness of the retarder may be adjusted according to the distribution of incident angles of the polarized light. It should be noted that in FIG. 10 as well, the polarization design for polarization conversion may be configured in reverse (reversing the S-polarized light and P-polarized light) from the above explanation.

[0141] In a typical TV device, the light emitted from the LCD panel 11 has similar diffusion characteristics in both the horizontal direction of the screen (shown on the X-axis in FIG. 12(a)) and the vertical direction of the screen (shown on the Y-axis in FIG. 12(b)). In contrast, the diffusion characteristics of the light beam emitted from the LCD panel of this embodiment are 1 / 5 of the 62-degree viewing angle of a typical TV device, as shown in Example 1 of FIG. 12, when the viewing angle at which the luminance is 50% of that at a front view (angle of 0 degrees) is set to 13 degrees. Similarly, the vertical viewing angle is asymmetric between the top and bottom, and the reflection angle and the area of ​​the reflective surface of the reflective light guide are optimized to keep the upper viewing angle to about 1 / 3 of the lower viewing angle. As a result, the amount of image light directed toward the monitoring direction is significantly improved compared to conventional LCD TVs, with brightness more than 50 times higher.

[0142] Furthermore, assuming the viewing angle characteristics shown in Example 2 in Figure 12, if the viewing angle at which brightness is 50% of that when viewed from the front (angle of 0 degrees) is set to 5 degrees, this will be 1 / 12 of the 62 degrees of devices used for general TV applications. Similarly, the vertical viewing angle is optimized by optimizing the reflection angle and the area of ​​the reflective surface of the reflective light guide so that the viewing angle is approximately 1 / 12 of that of devices used for general TV applications, with equal viewing angle both above and below. As a result, the amount of image light directed in the monitoring direction is significantly improved compared to conventional LCD TVs, and brightness is more than 100 times greater.

[0143] As described above, by setting the viewing angle to a narrow angle, the amount of luminous flux directed in the monitoring direction can be concentrated, significantly improving the efficiency of light utilization. As a result, even when using a liquid crystal display panel for general TV applications, by controlling the light diffusion characteristics of the light source device, it is possible to achieve a significant improvement in brightness with similar power consumption, making it possible to create a video display device that is compatible with information display systems facing bright outdoor environments.

[0144] When using a large LCD display panel, the overall brightness of the screen can be improved by directing the light from the periphery of the screen inward so that it is directed toward the observer when the observer is facing the center of the screen. Figure 11 shows the convergence angle between the long and short sides of the panel when the observer's distance from the panel, L, and the panel size (screen ratio 16:10) are used as parameters. When monitoring with the screen in portrait orientation, the convergence angle can be set to match the short side. For example, when using a 22" panel in portrait orientation and the monitoring distance is 0.8m, a convergence angle of 10 degrees will allow the image light from the four corners of the screen to be effectively directed toward the observer.

[0145] Similarly, when monitoring with a 15" panel in portrait orientation and the monitoring distance is 0.8 m, a convergence angle of 7 degrees will allow the image light from the four corners of the screen to be effectively directed towards the monitor. As described above, depending on the size of the LCD panel and whether it is used portrait or landscape, the overall brightness of the screen can be improved by directing the image light from the periphery of the screen towards the monitor who is in the optimum position to monitor the centre of the screen.

[0146] As shown in Figure 9, the basic configuration involves a light source device directing a light beam with a narrow angle of directionality to a liquid crystal display panel 11, which is then luminance-modulated according to a video signal. The video information displayed on the screen of the liquid crystal display panel 11 is then reflected by a retroreflector, and the resulting floating image is displayed indoors or outdoors via a transparent member 100.

[0147] By using the display device and light source device according to the embodiment of the present invention described above, it is possible to realize a space floating image display device with higher light utilization efficiency.

[0148] <Example of image display processing in a space floating image display device> Next, an example of a problem solved by the image processing of this embodiment will be described with reference to Fig. 13A. In the space-floating image display device 1000, when the far side of the space-floating image 3 from the user's perspective is inside the housing of the space-floating image display device 1000 and it is sufficiently dark, the user will visually recognize that the background of the space-floating image 3 is black.

[0149] Here, an example of displaying a character "panda" 1525 in the space floating image 3 will be described with reference to FIG. 13A. First, the image control unit 1160 in FIG. 3 determines a pixel area for drawing the image of the character "panda" 1525 and a background image as shown in FIG. 13A(1). For an image including a transparent information region 1520, the pixel region in which the image of the character "panda" 1525 is drawn is distinguished from the transparent information region 1520, which is the background image.

[0150] A method for distinguishing and recognizing the character image from the background image is, for example, to configure the image processing of the video control unit 1160 so that the background image layer and the character image layer in front of the background image layer can be processed as separate layers, and the character image and background image can be distinguished and recognized based on the superimposition relationship when these layers are combined.

[0151] Here, the image control unit 1160 recognizes the black of pixels that draw objects such as character images and transparent information pixels as different information. However, it is assumed that the brightness of both the black of pixels that draw objects and the transparent information pixels is 0. In this case, when the space floating image 3 is displayed, there is no difference in brightness between the pixels that draw black in the image of the character "panda" 1525 and the pixels of the transparent information area 1520, which is the background image. Therefore, in the space floating image 3, as shown in FIG. 13A(2), there is no brightness in either the pixels that draw black in the image of the character "panda" 1525 or the pixels of the transparent information area 1520, and they are visually recognized by the user as the same optically black space. In other words, the part that draws black in the image of the character "panda" 1525, which is an object, blends into the background, and the character Only the non-black parts of the "Panda" 1525 are recognized as floating images in the display area of ​​the Floating Image 3.

[0152] An example of image processing according to this embodiment will be described with reference to FIG. 13B. FIG. 13B is a diagram illustrating an example of image processing that more suitably resolves the issue of the black image region of the object blending into the background, as described in FIG. 13A. In FIGS. 13B(1) and 13B(2), the upper side shows the display state of the floating image 3 in space, and the lower side shows the input / output characteristics of the image processing of the image of the object. Note that the image of the object (character "panda" 1525) and the corresponding data may be read from the storage unit 1170 or memory 1109 in FIG. 3, or may be input from the video signal input unit 1131, or may be acquired via the communication unit 1132.

[0153] In the state shown in Figure 13B(1), the input / output characteristics of the image processing of the object image are in a linear state with no particular adjustment. In this case, the display state is the same as that shown in Figure 13A(2), and the black image area of ​​the object blends into the background. In contrast, in Figure 13B(2), the video control unit 1160 of this embodiment adjusts the input / output characteristics of the image processing of the image of the object (character "panda" 1525) to the input / output characteristics shown in the lower part.

[0154] That is, the video control unit 1160 performs image processing with input / output characteristics on the image of the object (character "panda" 1525), which has a characteristic of converting pixels in low-brightness areas of the input image into output pixels with increased brightness values. After the image of the object (character "panda" 1525) has been subjected to image processing with the input / output characteristics, a video including the image of the object (character "panda" 1525) is input to the display device 1 and displayed. Then, as shown in the upper part of FIG. 13B(2), the display state of the floating in space image 3 is such that the brightness of pixel areas depicting black in the image of character "panda" 1525 increases. This allows the user to distinguish the areas depicting black among the areas depicting the image of character "panda" 1525 without them blending into the black background, making it possible to display the object more appropriately.

[0155] 13B(2), the area displaying the image of the character "panda" 1525, which is an object, can be distinguished from the black background inside the housing of the space-floating image display device 1000 through the window, improving the visibility of the object. Therefore, for example, even if the object includes pixels with a brightness value of 0 before the image processing (i.e., when the image of the object or the corresponding data is read from the storage unit 1170 or memory 1109 in FIG. 3, or when the image of the object is input from the video signal input unit 1131, or when the data of the object is obtained via the communication unit 1132, etc.), the image processing of the input / output characteristics by the video control unit 1160 converts the object into an object with a brightness value of 0 increased for the pixels in the low-brightness area, and then the object is displayed on the display device 1 and converted into a space-floating image 3 by the optical system of the space-floating image display device 1000.

[0156] That is, the pixels that make up the object after image processing of the input / output characteristics are converted to a state in which they do not include pixels with a brightness value of 0, and then they are displayed on the display device 1 and converted into a floating image 3 in space by the optical system of the floating image display device 1000.

[0157] In the image processing of Figure 13B(2), a method for applying image processing with the input / output characteristics of Figure 13B(2) only to the image area of ​​the object (character "panda" 1525) is, for example, to configure the image processing of the video control unit 1160 so that the background image layer and the layer of the character image in front of the background image layer can be processed as separate layers, and the image processing with the input / output characteristics of Figure 13B(2) is applied to the character image layer, while not applying this image processing to the background image layer.

[0158] Then, by combining these layers, image processing with a characteristic of raising the low-brightness areas of the input image is performed only on the character image, as shown in Fig. 13B(2). Alternatively, after the character image layer and background image layer are combined, image processing with the input / output characteristics of Fig. 13B(2) may be performed only on the character image area.

[0159] Furthermore, the input / output image characteristics used in the image processing for boosting low-luminance regions of the input / output characteristics for the input image are not limited to the example shown in FIG. 13B(2). Any image processing for boosting low luminance may be used, including so-called brightness adjustment. Alternatively, image processing for improving visibility by controlling the gain that changes the weighting of Retinex processing, as disclosed in International Publication WO 2014 / 162533, may be performed.

[0160] According to the image processing of FIG. 13B(2) described above, it is possible to make the user aware of areas where black is drawn among areas where images of characters, objects, etc. are drawn without blending into the black background, thereby realizing a more suitable display.

[0161] 13A and 13B, the problems and more suitable image processing were explained using the space-floating image display device in which the background appears black (for example, the space-floating image display device 1000 in FIGS. 4A to 4G, or the space-floating image display device 1000 in the state where the rear window is shielded in FIGS. 4I and 4J). However, the image processing is also effective for devices other than these space-floating image display devices.

[0162] Specifically, in the space-floating image display device 1000 of Fig. 4H, or in Fig. 4I and Fig. 4J where the rear window is not shaded, the background of the space-floating image 3 is not black, but the scenery behind the space-floating image display device 1000 through the window. In this case, the same problems as those described in Fig. 13A and Fig. 13B exist.

[0163] That is, the part of the image of the character "panda" 1525, which is an object, that is drawn in black will blend into the scenery behind the space-floating image display device 1000 through the window. In this case too, by using the image processing of Fig. 13B(2), the part of the image of the character "panda" 1525, which is an object, that is drawn in black will be recognized as being distinct from the scenery behind the space-floating image display device 1000 through the window, improving the visibility of the object.

[0164] That is, by using the image processing of FIG. 13B(2), the area displaying the image of the object character "panda" 1525 can be recognized as distinct from the scenery behind the space floating image display device 1000 through the window, and it becomes possible to more easily recognize that the object character "panda" 1525 is in front of the scenery, improving the visibility of the object.

[0165] 4K, 4L, and 4M, as described above, when another image (such as an image from the transmissive self-luminous image display device 1650 or an image from the second display device 1680) is displayed at a position different in depth from the space-floating image 3, the background of the space-floating image 3 is not black, but the other image. In this case, the problems explained in FIGS. 13A and 13B also exist.

[0166] That is, the part of the image of the character "panda" 1525, which is an object, that is drawn in black will blend into the other image that is displayed at a different depth from the floating image in space 3. In this case as well, by using the image processing of Fig. 13B(2), the part of the image of the character "panda" 1525, which is an object, that is drawn in black will be able to be recognized as distinct from the other image, improving the visibility of the object.

[0167] In other words, by using the image processing of Figure 13B (2), the area displaying the image of the object character "panda" 1525 can be recognized as distinct from the other image, and it can be more easily recognized that the object character "panda" 1525 is in front of the other image, improving the visibility of the object.

[0168] An example of the image display process of this embodiment will be described with reference to Fig. 13C. Fig. 13C shows an example of the image display of this embodiment in which the space floating image 3 and a second image 2050, which is another image, are simultaneously displayed. The second image 2050 may correspond to the image displayed by the transmissive self-luminous image display device 1650 of Fig. 4K or Fig. 4L. The second image 2050 may also correspond to the image displayed by the second display device 1680 of Fig. 4M.

[0169] That is, the example of the image display in Fig. 13C shows a specific example of the image display examples of the space-floating image display device 1000 in Figs. 4K, 4L, and 4M. In the example of this figure, a bear character is displayed in the space-floating image 3. The area other than the bear character in the space-floating image 3 is displayed in black, and becomes transparent as a space-floating image. In addition, the second image 2050 is a background image in which a plain, a mountain, and a sun are drawn.

[0170] 13C, the floating in space image 3 and the second image 2050 are displayed at different depth positions. When the user 230 views the two images, the floating in space image 3 and the second image 2050, in the line of sight of the arrow 2040, the user 230 can view the two images in a superimposed state. Specifically, the bear character of the floating in space image 3 appears to be superimposed in front of the background of plains, mountains, and the sun depicted in the second image 2050.

[0171] Here, since the space floating image 3 is formed as a real image in the air, when the user 230 moves his / her viewpoint slightly, he / she can recognize the depth of the space floating image 3 and the second image 2050 due to parallax. Therefore, the user 230 can get a stronger sense of floating in space from the space floating image 3 while viewing the two images in an overlapping state.

[0172] An example of the video display process of this embodiment will be described with reference to Fig. 13D. Fig. 13D(1) is a diagram of the floating in space image 3, from the example of the video display of this embodiment in Fig. 13C, as seen from the line of sight of the user 230. Here, a bear character is displayed in the floating in space image 3. The area other than the bear character in the floating in space image 3 is displayed in black, and becomes transparent as a floating in space image.

[0173] 13D(2) is a diagram showing the second image 2050 in the example of the video display of this embodiment in FIG. 13C as viewed from the line of sight of the user 230. In the example of this figure, the second image 2050 is a background image in which a plain, a mountain, and a sun are drawn.

[0174] 13D(3) is a diagram showing the state in which the second image 2050 and the floating in space image 3 appear superimposed in the line of sight of the user 230, among the example of image display of this embodiment in Fig. 13C. Specifically, the bear character of the floating in space image 3 appears superimposed in front of the background of plains, mountains, and the sun drawn in the second image 2050.

[0175] Here, when simultaneously displaying the space-floating image 3 and the second image 2050, it is desirable to pay attention to the balance of brightness between the two images in order to ensure better visibility of the space-floating image 3. If the second image 2050 is too bright compared to the brightness of the space-floating image 3, the displayed image of the space-floating image 3 will be transparent, and the second image 2050, which is the background, will be strongly visible through it.

[0176] Therefore, the output of the light source of the spatially floating image 3 and the display image brightness of the display device 1, and the output of the light source of the display device displaying the second image 2050 and the display image brightness of the display device should be set so that at least the brightness per unit area of ​​the spatially floating image 3 at the display position of the spatially floating image 3 is greater than the brightness per unit area of ​​the image light that reaches the display position of the spatially floating image 3 from the second image 2050.

[0177] Note that this condition only needs to be satisfied when simultaneously displaying the space-floating image 3 and the second image 2050, and therefore when switching from the first display mode in which only the second image 2050 is displayed without displaying the space-floating image 3 to the second display mode in which the space-floating image 3 and the second image 2050 are simultaneously displayed, control may be performed to reduce the brightness of the second image 2050 by lowering the output of the light source of the display device that displays the second image 2050 and / or the display image brightness of the display device. These controls may be realized by the control unit 1110 in Fig. 3 controlling the display device 1 and the display device that displays the second image 2050 (the transmissive self-luminous image display device 1650 in Fig. 4K or Fig. 4L or the second display device 1680 in Fig. 4M).

[0178] Note that when switching from the above-described first display mode to the above-described second display mode, if control is performed to reduce the brightness of the second image 2050, the brightness may be reduced uniformly across the entire screen of the second image 2050. Alternatively, instead of reducing the brightness uniformly across the entire screen of the second image 2050, the brightness reduction effect may be greatest in the portion where an object is displayed in the space-floating image 3, and the brightness reduction effect may be gradually weakened around that portion. In other words, the visibility of the space-floating image 3 can be sufficiently ensured by reducing the brightness of the second image 2050 only in the portion where the space-floating image 3 is visually recognized as being superimposed on the second image 2050.

[0179] Here, since the space floating image 3 and the second image 2050 are displayed at positions with different depths, when the user 230 slightly changes his / her viewpoint, the parallax causes a change in the superimposed position of the space floating image 3 relative to the second image 2050. Therefore, when switching from the above-mentioned first display mode to the above-mentioned second display mode, if the brightness is to be reduced unevenly across the entire screen of the second image 2050, it is not desirable to reduce the brightness sharply based on the outline of the object displayed in the space floating image 3, but rather it is desirable to perform a gradation process of the brightness reduction effect, which changes the brightness reduction effect stepwise depending on the position as described above.

[0180] In addition, in the space floating image display device 1000 where the position of the object displayed in the space floating image 3 is approximately at the center of the space floating image 3, the position where the brightness reduction effect of the gradation processing of the brightness reduction effect is greatest can be the center position of the space floating image 3.

[0181] According to the image display process of this embodiment described above, the user 230 can visually recognize the space floating image 3 and the second image 2050 more favorably.

[0182] Note that when displaying the space-floating image 3, control may be performed so as not to display the second image 2050. Since not displaying the second image 2050 increases the visibility of the space-floating image 3, this is suitable for applications such as the space-floating image display device 1000 where the user must be able to reliably view the space-floating image 3 when the space-floating image 3 is displayed.

[0183] <Example 2> As Example 2 of the present invention, an example of another configuration example of the space-floating image display device will be described. Note that the space-floating image display device according to this example is obtained by changing the optical system stored in the space-floating image display device described in Example 1 to the optical system shown in FIG. 14(1) or FIG. 14(2). In this example, differences from Example 1 will be described, and repeated explanations of the same configuration as Example 1 will be omitted. Note that in the following description of this example, the predetermined polarized light and the other polarized light are polarized waves whose phases differ by 90° from each other.

[0184] Fig. 14(1) shows an example of an optical system and an optical path according to this embodiment. The optical system shown in Fig. 14(1) is configured such that the display device 1 is closer to the polarization separation member 101B in the optical system of Fig. 2C, making the entire optical system more compact. In Fig. 14(1), the components denoted by the same reference numerals as in Fig. 2C will not be described in detail again.

[0185] 14(1), similar to FIG. 2C, image light of a predetermined polarized light (P polarized light in the figure) emitted from display device 1 travels in a direction perpendicular to the image display surface of display device 1. Here, similar to FIG. 2C, polarization separation member 101B selectively transmits the predetermined polarized light (P polarized light in the figure) emitted from display device 1 and reflects the other polarized light (S polarized light in the figure).

[0186] Therefore, image light of a predetermined polarization (P-polarized light in the figure) traveling vertically from the image display surface of display device 1 passes through polarization separator 101B and reaches retroreflector 2 to which λ / 4 plate 21 is attached. The image light that is retroreflected by retroreflector 2 and travels again toward polarization separator 101B has passed through λ / 4 plate 21 twice, and is converted from the predetermined polarization (P-polarized light in the figure) at the time of emission from display device 1 to the other polarization (S-polarized light in the figure). The image light that travels again toward polarization separator 101B is the other polarization (S-polarized light in the figure), and is therefore reflected by polarization separator 101B toward the position where the user should be. The traveling direction of the image reflected by polarization separator 101B is determined based on the angle at which polarization separator 101B is disposed.

[0187] In the example of Figure 14(1), the image light traveling toward the polarization separation member 101B is reflected at a right angle by the polarization separation member 101B and travels as shown in the figure. The image light reflected by the polarization separation member 101B forms a space-floating image 3A. The space-floating image 3A can be viewed by the user from the direction of arrow A.

[0188] Here, due to the characteristics of retroreflection by the retroreflector 2, the optical path length of the image light emitted from the display device 1 until it reaches the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 until it reaches the formation position of the space floating image 3A. This relationship determines the formation position of the space floating image 3A in the traveling direction of the image light reflected by the polarization separation member 101B.

[0189] In the example of FIG. 14(1), the display device 1, the polarization separating member 101B, and the retroreflector 2 are arranged closer than in the example of FIG. 2C. This allows the entire optical system to be configured more compactly. However, the amount by which the space-floating image 3A protrudes from the optical system of FIG. 14(1) is not very large. For example, as an index of the amount by which the space-floating image 3A protrudes from the optical system, the figure shows the distance from the position where the central light beam of the image light is reflected by the polarization separating member 101B to the position where the image light forms the space-floating image 3A (L1 in the example of FIG. 14(1)).

[0190] 14(1), the characteristics of P polarization and S polarization may be interchanged. Specifically, the predetermined polarization of the image light emitted from the display device 1 may be S polarization, and the characteristics of P polarization and S polarization may be interchanged in the reflection characteristics of the polarization separation member 101B. In this case, the P polarization and S polarization shown in the figure are both reversed, but the optical design, such as the optical path, can be realized in exactly the same way.

[0191] Next, Fig. 14(2) shows another example of an optical system and an optical path according to this embodiment. The optical system of Fig. 14(2) is a modified version of the optical system of Fig. 14(1) in order to increase the amount of the floating image projecting from the optical system while realizing the same compactness as the optical system of Fig. 14(1). In Fig. 14(2), the components with the same reference numerals as those in Fig. 14(1) will not be described in detail again.

[0192] 14(2), similar to FIG. 14(1), image light of a predetermined polarized light (P polarized light in the figure) emitted from the display device 1 travels in a direction perpendicular to the image display surface of the display device 1. Here, the polarization characteristics of the polarization separation member 101B are arranged 90 degrees differently from those in FIG. 14(1). Image light of a predetermined polarized light (P polarized light in the figure) traveling in a direction perpendicular to the image display surface of the display device 1 passes through the polarization separation member 101B.

[0193] 14(1), the image light passing through the polarization separation member 101B is not faced with the retroreflector 2 having the λ / 4 plate 21 attached thereto, but with the specular reflector 4 having the λ / 4 plate 21B attached thereto. Here, the reflection at the specular reflector 4 is specular reflection (also called regular reflection), not retroreflection.

[0194] Therefore, the image light that has passed through polarization separation member 101B is specularly reflected by specular reflector 4 to which λ / 4 plate 21B is attached. The image light that has been specularly reflected by specular reflector 4 and travels again toward polarization separation member 101B has been converted from the predetermined polarization (P polarization in the figure) at the time of emission from display device 1 to the other polarization (S polarization in the figure) by having passed through λ / 4 plate 21 twice. The image light that has traveled again toward polarization separation member 101B is the other polarization (S polarization in the figure), and is therefore reflected by polarization separation member 101B.

[0195] Here, because the orientation of the polarization separator 101B in Figure 14(2) is different from that in Figure 14(1), the image light reflected by the polarization separator 101B travels in the opposite direction from where the user should be. A retroreflector 2 with a λ / 4 plate 21C attached is disposed at the destination of the image light reflected by the polarization separator 101B. The image light is retroreflected by the retroreflector 2. The image light that is retroreflected by the retroreflector 2 and travels again toward the polarization separator 101B has been converted from the other polarized light (S-polarized light in the figure) to the specified polarized light (P-polarized light in the figure) by passing through the λ / 4 plate 21C twice.

[0196] The image light that travels back toward the polarization separation member 101B is of a predetermined polarization (P polarization in the figure), so it passes through the polarization separation member 101B and continues toward the location where the user should be. The image light that has passed through the polarization separation member 101B forms a space-floating image 3B. The space-floating image 3B can be easily viewed by the user from the direction of arrow A.

[0197] 14(2), similarly to FIG. 14(1), due to the characteristics of retroreflection by the retroreflector 2, the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 to reach the formation position of the space-floating image 3B. This relationship determines the formation position of the space-floating image 3B in the traveling direction of the image light transmitted through the polarization separation member 101B.

[0198] 14(2), the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 is longer than the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 in FIG. 14(1). This is because in the optical system of FIG. 14(2), an optical path that goes back and forth between the polarization separation member 101B and the specular reflector 4, which does not exist in the optical system of FIG. 14(1), is added to the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2.

[0199] As a result, in the optical system of Figure 14(2), the distance from the position where the central ray of the image light passes through the polarization separation member 101B to the position where the image light forms the space-floating image 3B (L2 in the example of Figure 14(2)) is significantly longer than the distance from the position where the central ray of the image light is reflected by the polarization separation member 101B to the position where the image light forms the space-floating image 3A (L1 in the example of Figure 14(1)) in the optical system of Figure 14(1).

[0200] 14(2), the characteristics of P-polarized light and S-polarized light may be interchanged. Specifically, the predetermined polarization of the image light emitted from the display device 1 may be S-polarized light, and the characteristics of P-polarized light and S-polarized light may be interchanged with respect to the reflection characteristics of the polarization separation member 101B. In this case, the P-polarized light and S-polarized light shown in the figure are both reversed, but the optical design, such as the optical path, can be realized in exactly the same way.

[0201] As described above, according to the optical systems of Fig. 14(1) and Fig. 14(2) in the second embodiment of the present invention, a more compact optical system can be realized. In particular, according to the optical system of Fig. 14(2), it is possible to increase the amount of the floating image projecting from the optical system, despite the more compact optical system.

[0202] When the optical system of Fig. 14(1) or Fig. 14(2) is incorporated into a space-floating image display device, it can be realized by replacing the optical system in the space-floating image display device described in Example 1 with the optical system of Fig. 14(1) or Fig. 14(2). Specifically, the optical system of Fig. 14(1) may be replaced with the optical system of the space-floating image display device of Fig. 4E, Fig. 4F, Fig. 4G, Fig. 4H, Fig. 4I, Fig. 4J, Fig. 4K, Fig. 4L, or Fig. 4M. In this case, since the optical system becomes compact, it is possible to make the housing of the space-floating image display device of each figure smaller.

[0203] Specifically, the optical system of Fig. 14(2) may be replaced with the optical system of the space-floating image display device of Fig. 4E, Fig. 4F, Fig. 4G, Fig. 4K, or Fig. 4L. In this case, it becomes possible to increase the amount of space-floating images projecting from the optical system. Also, since the optical system becomes compact, it is possible to make the housing of the space-floating image display device of each figure smaller.

[0204] Example 3 As a third embodiment of the present invention, a space-floating image display device that displays a plurality of layers of space-floating images with different amounts of projection from an optical system will be described as an example of another configuration example of a space-floating image display device. Note that in this embodiment, differences from the first or second embodiment will be described, and repeated explanations of the same configuration as the first or second embodiment will be omitted. Note that in the following description of this embodiment, the predetermined polarized light and the other polarized light are polarized waves whose phases are different from each other by 90°.

[0205] Fig. 15A shows an example of the configuration and optical path of an optical system of a space-floating image display device that displays multiple layers of space-floating images. The optical system of Fig. 15A has only one display device 1 as the display device, which is the image source. In the example of Fig. 15A, two display areas, display area 1501 and display area 1502, are provided on the display screen of display device 1. The optical system of Fig. 15A displays space-floating image 3D corresponding to display area 1501. The optical system of Fig. 15A displays space-floating image 3E corresponding to display area 1502.

[0206] In the example of Fig. 15A, when a user views the space floating image 3D and the space floating image 3E from the direction of arrow A, the space floating image 3D appears to be displayed in front of the space floating image 3E. Since the space floating image 3D and the space floating image 3E appear to overlap from the user's perspective, these space floating images are viewed as a space floating image with two layers of depth.

[0207] Next, we will explain the detailed configuration of the optical system in Fig. 15A that realizes this spatial floating image with two layers of depth. The configuration of the display device 1 is the same as in Example 1, so repeated explanation will be omitted. First, image light of a predetermined polarization (P polarization in the figure) is output from the display device 1. Image light of a predetermined polarization (P polarization in the figure) is output at both positions in the display area 1501 and the display area 1502, but in the optical system of Fig. 15A, the λ / 2 plate 22 is attached so as to include the display area 1502, so the image light emitted from the display area 1502 passes through the λ / 2 plate 22 and is converted to the other polarization (S polarization in the figure) before traveling.

[0208] Here, image light of a predetermined polarized light (P polarized light in the figure) output from display region 1501 travels as shown in the figure and enters polarization separation member 101D. Polarization separation member 101D selectively transmits the predetermined polarized light (P polarized light in the figure) and reflects the other polarized light (S polarized light in the figure).

[0209] Therefore, the image light of a predetermined polarization (P polarization in the figure) output from the display area 1501 passes through the polarization separation member 101D and reaches the retroreflector 2D to which the λ / 4 plate 21D is attached. The image light that is retroreflected by the retroreflector 2D and travels again toward the polarization separation member 101D has been converted from the predetermined polarization (P polarization in the figure) at the time of output from the display device 1 to the other polarization (S polarization in the figure) by passing through the λ / 4 plate 21D twice.

[0210] The image light that travels back toward the polarization separator 101D is the other polarized light (S-polarized light in the figure), and is therefore reflected by the polarization separator 101D toward the location where the user should be. The traveling direction of the image reflected by the polarization separator 101D is determined based on the angle at which the polarization separator 101D is placed. In the example of FIG. 15A, the image light that travels toward the polarization separator 101D is reflected at a right angle by the polarization separator 101D and travels as shown in the figure. The image light reflected by the polarization separator 101D forms a space-floating image 3D.

[0211] Next, the image light output from the display region 1502 passes through the λ / 2 plate 22, is converted into the other polarized light (S-polarized light in the figure), and travels onward, entering the polarization separation member 101E. The polarization separation member 101E selectively transmits a predetermined polarized light (P-polarized light in the figure) and reflects the other polarized light (S-polarized light in the figure). Therefore, the image light of the other polarized light (S-polarized light in the figure) output from the display region 1502 and transmitted through the λ / 2 plate 22 is reflected by the polarization separation member 101E and reaches the retroreflector 2E to which the λ / 4 plate 21E is attached. The traveling direction of the image reflected by the polarization separation member 101E is determined based on the angle at which the polarization separation member 101E is disposed. In the example of FIG. 15A, the image light traveling toward the polarization separation member 101E is reflected at a right angle by the polarization separation member 101E and travels as shown in the figure. The image light that is retroreflected by the retroreflector 2E and travels again toward the polarization separation member 101E is converted from the other polarization (S polarization in the figure) to the specified polarization (P polarization in the figure) by passing through the λ / 4 plate 21E twice.

[0212] The image light that travels back toward the polarization separation member 101E is of a predetermined polarization (P polarization in the figure), and therefore passes through the polarization separation member 101E. As shown in the figure, the image light that has passed through the polarization separation member 101E travels toward the polarization separation member 101D. As described above, the polarization separation member 101D selectively transmits light of a predetermined polarization (P polarization in the figure) and reflects the other polarization (S polarization in the figure). Therefore, the image light from the retroreflector 2E, which is of a predetermined polarization (P polarization in the figure), passes through the polarization separation member 101D and travels toward the position where the user should be. The image light that has passed through the polarization separation member 101D forms the spatially floating image 3E.

[0213] In the example of Figure 15A, a light shielding plate is provided between the optical path of the image light output from display area 1501 and the optical path of the image light output from display area 1502 to prevent each image light from leaking into the optical path of the other image light.

[0214] In the example of FIG. 15A, the polarization separation member 101D and the polarization separation member 101E are both arranged at an angle of 45 degrees with respect to the traveling direction of the image light from the display device 1. As a result, the image light forming the space floating image 3D and the image light forming the space floating image 3E travel in the same direction toward the position where the user should be. In order to configure it in this way, in the example of FIG. 15A, the user must move in the direction of arrow A (y direction) to view the space floating image 3D and the space floating image 3E. When the user views the image 3E, the floating image 3D, the floating image 3E, the polarization separation member 101D, the polarization separation member 101E, and the retroreflector 2E are arranged on the same straight line as seen by the user (for example, in the example of Figure 15A, this is the straight line of the optical path from the retroreflector 2E to the floating image 3E, which extends in the direction of the user).

[0215] In this case, the display device 1 and the retroreflector 2D are positioned away from the collinear position. In the example of FIG. 15A, the polarization separator 101D and the polarization separator 101E are positioned so that when a user views the space-floating image 3D and the space-floating image 3E from the direction of arrow A (y direction), the horizontal center (x direction) of the image of the space-floating image 3D coincides with the horizontal center (x direction) of the image of the space-floating image 3E. If the horizontal center (x direction) of the image of the space-floating image 3D coincides with the horizontal center (x direction) of the image of the space-floating image 3E, this is more suitable for the user, as it is easier to view for the user, and video content creators do not need to consider offsets. This also simplifies the optical layout, making it more suitable.

[0216] 15A, the characteristics of P polarization and S polarization may be interchanged. Specifically, a predetermined polarization of the image light emitted from display region 1501 of display device 1 may be S polarization, and the other polarization emitted from display region 1502 of display device 1 and transmitted through λ / 2 plate 22 may be P polarization, and the reflection characteristics of polarization separator 101D and polarization separator 101E may have the P polarization and S polarization characteristics interchanged. In this case, the illustrated P polarization and S polarization are both reversed, but the optical design, such as the optical path, can be realized in exactly the same way.

[0217] According to the optical system of FIG. 15A described above, an optical system that forms a floating image with two layers of depth can be realized using a single display device. Note that a configuration may also be adopted in which separate display devices are provided for display area 1501 and display area 1502. However, a configuration that includes multiple display devices will require additional corresponding circuits, which may result in relatively high costs. Therefore, if a floating image with two layers of depth is formed using only a single display device as in FIG. 15A, an optical system that forms a floating image with two layers of depth can be realized at lower cost.

[0218] Note that when the optical system of FIG. 15A is incorporated into a space-floating image display device, it can be realized by replacing the optical system in the space-floating image display device described in Example 1 with the optical system of FIG. 15A. Specifically, the optical system of FIG. 15A may be replaced with the optical system of the space-floating image display device of FIG. 4E, FIG. 4F, FIG. 4G, FIG. 4K, or FIG. 4L. In this case, a space-floating image display device that forms a space-floating image with two layers of depth in each figure can be realized. In particular, in FIG. 4K and FIG. 4L, a space-floating image with two layers of depth can be formed on the front side as seen from the user of the transmissive self-luminous image display device 1650. In this case, a space-floating image with two layers of depth and a three-layer image with different depths can be formed so that they can be viewed by the user on the transmissive self-luminous image display device 1650.

[0219] Next, another example of the optical system and optical path of the space-floating image display device that displays multiple layers of space-floating images with different protrusion amounts from the optical system of Example 3 will be described with reference to Fig. 15B. The optical system of Fig. 15B is a modified example in which a part of the configuration of the optical system of Fig. 15A is changed. Therefore, in the example of Fig. 15B, differences from Fig. 15A will be described, and repeated explanations of the same configuration as Fig. 15A will be omitted.

[0220] In the optical system of FIG. 15B, similar to FIG. 15A, two display areas, display area 1501 and display area 1502, are provided on the display screen of the display device 1. However, no λ / 2 plate is attached to the exit surface of display area 1502. The characteristics and arrangement of polarization separation member 101D and polarization separation member 101E in the optical system of FIG. 15B are similar to those of the optical system of FIG. 15A. In the optical system of FIG. 15A, retroreflector 2D to which λ / 4 plate 21D is attached and retroreflector 2E to which λ / 4 plate 21E is attached are separately arranged. In contrast, the optical system of FIG. 15B has only one retroreflector 2 to which λ / 4 plate 21 is attached. In the optical system of FIG. 15B, a λ / 2 plate 22 is arranged in the optical path from polarization separation member 101E to polarization separation member 101D.

[0221] Here, in the optical system of Figure 15B, the optical path of the image light of a predetermined polarization (P polarization in the figure) emitted from the display area 1501 until it forms the spatially floating image 3D, and the optical characteristics of each optical element are the same except that the retroreflector 2D to which the λ / 4 plate 21D is attached is replaced by the retroreflector 2 to which the λ / 4 plate 21 is attached, so explanation will be omitted.

[0222] 15B, a predetermined polarized light (P polarized light in the figure) emitted from the display region 1502 travels toward the polarization separation member 101E and is incident on the polarization separation member 101E. The polarization separation member 101E selectively transmits the predetermined polarized light (P polarized light in the figure) and reflects the other polarized light (S polarized light in the figure). Therefore, the image light traveling toward the polarization separation member 101E is transmitted through the polarization separation member 101E and travels toward the retroreflector 2. The image light that is retroreflected by the retroreflector 2 and travels again toward the polarization separation member 101E has been converted from the predetermined polarized light (P polarized light in the figure) to the other polarized light (S polarized light in the figure) by passing through the λ / 4 plate 21 twice.

[0223] The image light that travels back toward the polarization separation member 101E is the other polarization (S-polarized light in the figure), so it is reflected by the polarization separation member 101E and travels toward the λ / 2 plate 22. The image light that enters the λ / 2 plate 22 is converted from the other polarization (S-polarized light in the figure) to the specified polarization (P-polarized light in the figure) by passing through the λ / 2 plate 22. The image light that has passed through the λ / 2 plate 22 travels toward the polarization separation member 101D. The polarization separation member 101D selectively transmits the specified polarization (P-polarized light in the figure) and reflects the other polarization (S-polarized light in the figure). Therefore, the image light from the λ / 2 plate 22, which is the specified polarization (P-polarized light in the figure), passes through the polarization separation member 101D and travels toward the position where the user is supposed to be. The image light that has passed through the polarization separation member 101D forms the space-floating image 3E.

[0224] In the example of Figure 15B, a light shielding plate is provided between the optical path of the image light output from display area 1501 and the optical path of the image light output from display area 1502 to prevent each image light from leaking into the optical path of the other image light.

[0225] 15B, the polarization separation members 101D and 101E are both arranged at an angle of 45 degrees to the traveling direction of the image light from the display device 1. As a result, the image light forming the space floating image 3D and the image light forming the space floating image 3E travel in the same direction toward the position where the user should be.

[0226] To achieve this configuration, in the example of FIG. 15B, the user must move in the direction of arrow A (y direction). When the user visually recognizes the space-floating image 3D and the space-floating image 3E, the space-floating image 3D, the space-floating image 3E, the polarization separation member 101D, the λ / 2 plate 22, and the polarization separation member 101E are arranged on the same straight line as seen from the user (for example, in the example of FIG. 15B, the straight line of the optical path from the polarization separation member 101E to the space-floating image 3E, which extends in the direction of the user). In addition, at this time, the display device 1 and the retroreflector 2 are arranged in a position that is off the same straight line.

[0227] In the example of FIG. 15B, the user moves in the direction of arrow A (y direction) and sees the floating image 3D and the sky. When viewing the Space Floating Image 3E, the center of the left and right direction (x direction) of the Space Floating Image 3D image is , the polarization separation is performed so that the center of the image of the image floating in space 3E in the left-right direction (x direction) coincides with the center of the image of the image floating in space 3E. The installation positions of the member 101D and the polarization separation member 101E are determined. If the center of the left-right direction (x direction) of the image of the space floating image 3D and the center of the left-right direction (x direction) of the image of the space floating image 3E coincide with each other from the user's perspective, it is more preferable because it is easier for the user to see and the video content producers do not need to consider offsets. In addition, the optical layout becomes simpler, which is more preferable.

[0228] 15B, the characteristics of P polarization and S polarization may be interchanged. Specifically, the predetermined polarization of the image light emitted from display region 1501 of display device 1 may be S polarization, the predetermined polarization of the image light emitted from display region 1502 may be S polarization, and the reflection characteristics of polarization separator 101D and polarization separator 101E may have the P polarization and S polarization characteristics interchanged. In this case, the illustrated P polarization and S polarization are both reversed, but the optical design, such as the optical path, can be realized in exactly the same way.

[0229] In the optical system of Fig. 15B described above, the optical path length from the image light emitted from the display area 1501 to form the space-floating image 3D is the same as that of the optical system of Fig. 15A. Also, in the optical system of Fig. 15B, the optical path length from the image light emitted from the display area 1502 to form the space-floating image 3E is the same as that of the optical system of Fig. 15A. Therefore, the positions where the space-floating image 3D and the space-floating image 3E are formed are also the same as those of the optical system of Fig. 15A.

[0230] In the example of FIG. 15B, the optical path length of the image light emitted from display area 1501 to form space-floating image 3D is the same as the optical path length of the image light emitted from display area 1502 to form space-floating image 3E. While retroreflectors 2D and 2E are arranged separately in the optical system of FIG. 15A, the optical system of FIG. 15B is composed of a single retroreflector 2. Because retroreflectors are components with high processing costs, reducing the cost to one piece contributes to cost reduction. Therefore, the optical system of FIG. 15B can realize an optical system that forms a space-floating image with two layers of depth at a lower cost than the optical system of FIG. 15A.

[0231] Next, another example of the optical system and optical path of the space-floating image display device that displays multiple layers of space-floating images with different protrusion amounts from the optical system of Example 3 will be described with reference to Fig. 16A. The optical system of Fig. 16A is a modified example in which a part of the configuration of the optical system of Fig. 15A is changed. Therefore, in the example of Fig. 16A, differences from Fig. 15A will be described, and repeated explanations of the same configuration as Fig. 15A will be omitted.

[0232] In the optical system of FIG. 16A , the display device 1 is tilted relative to the optical system of FIG. 15A around a position between display areas 1501 and 1502 on the display screen of the display device 1. In the example shown, the tilt of the display device 1 is 30 degrees. In the example shown, the length of the display screen of the display device 1 is increased in accordance with the tilt. The tilt of the display device 1 is determined so that the optical path length from the image light emitted from the display area 1501 to the retroreflector 2D is shorter than that of the optical system of FIG. 15A. As a result, the floating-in-space image 3D is formed further back from the user's perspective than in the optical system of FIG. 15A. Furthermore, the tilt of the display device 1 makes the optical path length from the image light emitted from the display area 1502 to the retroreflector 2E longer than that of the optical system of FIG. 15A. The floating-in-space image 3E is formed closer to the user's perspective than in the optical system of FIG. 15A.

[0233] Therefore, in the optical system of Fig. 16A, the distance in the depth direction between the two-layered space-floating images, Space Floating Image 3D and Space Floating Image 3E, can be made closer than in the optical system of Fig. 15A. That is, in the example of Fig. 16A, due to the tilt of the display device 1, the optical path length from the image light emitted from the display area 1501 to the retroreflector 2D is shorter than the optical path length from the image light emitted from the display area 1502 to the retroreflector 2E. Note that, by tilting the display device 1 with respect to the arrangement of the optical system of Fig. 15A, the two-layered space-floating images of the formed Space Floating Image 3D and Space Floating Image 3E are both tilted with respect to the arrangement of the optical system of Fig. 15A. When the tilt of the display device 1 with respect to the optical system of Fig. 15A is 30 degrees, the tilt of the two-layered space-floating images, Space Floating Image 3D and Space Floating Image 3E, is also 30 degrees.

[0234] In the optical system of FIG. 16A , an angle control sheet 23 may be attached to the surface of the absorptive polarizer 12 to accommodate the tilted position of the display device 1. The angle control sheet 23 is a sheet that controls the traveling direction of light so as to shift it by a predetermined angle. Specifically, it can be realized by a linear Fresnel lens sheet. In the angular distribution of the image light from the liquid crystal display panel 11, if the light intensity is strongest in the normal direction to the surface of the liquid crystal display panel 11, the traveling angle of the light can be controlled to offset the tilt of the display device 1, thereby improving the light utilization efficiency of the optical system.

[0235] If the tilt of display device 1 is 30 degrees, angle control sheet 23 that changes the angle of light travel by 30 degrees can be used to offset the tilt. When angle control sheet 23 is used, λ / 2 plate 22 can be attached to the surface of angle control sheet 23 for display area 1502. Note that angle control sheet 23 only needs to be used when it is necessary to improve the light utilization efficiency of the optical system, and the optical system of FIG. 16A can be configured without using angle control sheet 23.

[0236] In the optical system of FIG. 16A, the optical path of the image light emitted from display area 1501 and display area 1502 until it forms space-floating image 3D and space-floating image 3E, and the details of each optical element are the same as those of the optical system of FIG. 15A, so repeated explanations will be omitted.

[0237] 16A, the characteristics of P polarization and S polarization may be interchanged. Specifically, a predetermined polarization of the image light emitted from display region 1501 of display device 1 may be S polarization, and the other polarization emitted from display region 1502 of display device 1 and transmitted through λ / 2 plate 22 may be P polarization, and the reflection characteristics of polarization separator 101D and polarization separator 101E may have the P polarization and S polarization characteristics interchanged. In this case, the illustrated P polarization and S polarization are both reversed, but the optical design, such as the optical path, can be realized in exactly the same way.

[0238] In the example of FIG. 16A, the user also moves the floating image 3 in the direction of the arrow A (y direction). When the user visually recognizes the image 3D and the space-floating image 3E, the image 3D, the image 3E, the polarization separation member 101D, the polarization separation member 101E, and the retroreflector 2E are arranged on the same straight line as seen from the user (for example, in the example of FIG. 16A, the straight line of the optical path from the retroreflector 2E to the space-floating image 3E, which extends in the direction of the user). In addition, at this time, the display device 1 and the retroreflector 2D are arranged in a position that is off the same straight line.

[0239] By using the optical system of FIG. 16A explained above, it is possible to realize a space floating image display device in which the distance in the depth direction between the two layers of space floating images is made closer.

[0240] Next, another example of the optical system and optical path of the space-floating image display device that displays multiple layers of space-floating images with different protrusion amounts from the optical system of Example 3 will be described with reference to Fig. 16B. The optical system of Fig. 16B is a modified example in which a part of the configuration of the optical system of Fig. 15B is changed. Therefore, in the example of Fig. 16B, differences from Fig. 15B will be described, and repeated explanations of the same configuration as Fig. 15B will be omitted.

[0241] In the optical system of FIG. 16B , the display device 1 is tilted relative to the optical system of FIG. 15B around a position between display areas 1501 and 1502 on the display screen of the display device 1. In the example shown in the figure, the tilt of the display device 1 is 30 degrees. In the example shown in the figure, the length of the display screen of the display device 1 is increased in accordance with the tilt. The tilt of the display device 1 is determined so that the optical path length from the image light emitted from the display area 1501 to the retroreflector 2 is shorter than that of the optical system of FIG. 15B. As a result, the floating-in-space image 3D is formed further back from the user's perspective than in the optical system of FIG. 15B. Furthermore, the tilt of the display device 1 is determined so that the optical path length from the image light emitted from the display area 1502 to the retroreflector 2 is longer than that of the optical system of FIG. 15B. The floating-in-space image 3E is formed closer to the user's perspective than in the optical system of FIG. 15A.

[0242] Therefore, in the optical system of Fig. 16B, the distance in the depth direction between the two layers of space-floating images, space-floating image 3D and space-floating image 3E, can be made closer than in the optical system of Fig. 15B. That is, in the example of Fig. 16B, due to the tilt of the display device 1, the optical path length from the image light emitted from the display area 1501 to the retroreflector 2 is shorter than the optical path length from the image light emitted from the display area 1502 to the retroreflector 2.

[0243] Furthermore, since the display device 1 is arranged at an angle relative to the arrangement of the optical system in Fig. 15B, the two-layered space-floating images, space-floating image 3D and space-floating image 3E, that are formed are both arranged at an angle relative to the arrangement of the optical system in Fig. 15B. When the inclination of the display device 1 relative to the optical system in Fig. 15B is 30 degrees, the inclination of the two-layered space-floating images, space-floating image 3D and space-floating image 3E, is also 30 degrees.

[0244] Here, in the optical system of FIG. 16B , as with the optical system of FIG. 16A , an angle control sheet 23 may be attached to the surface of the absorptive polarizer 12 to accommodate the tilted arrangement of the display device 1. In the angular distribution of image light from the liquid crystal display panel 11, if the light intensity is strongest in the direction normal to the surface of the liquid crystal display panel 11, the light utilization efficiency of the optical system can be improved by controlling the light propagation angle to offset the tilt of the display device 1. If the tilt of the display device 1 is 30 degrees, an angle control sheet 23 that changes the light propagation angle by 30 degrees can be used to offset the tilt. Note that the angle control sheet 23 may be used only when it is necessary to improve the light utilization efficiency of the optical system; the optical system of FIG. 16B can be configured without the angle control sheet 23.

[0245] In the optical system of FIG. 16B, the optical path of the image light emitted from display area 1501 and display area 1502 until it forms space-floating image 3D and space-floating image 3E, and the details of each optical element are the same as those of the optical system of FIG. 15A, so repeated explanations will be omitted.

[0246] 16B, the characteristics of P polarization and S polarization may be interchanged. Specifically, the predetermined polarization of the image light emitted from display region 1501 of display device 1 may be S polarization, the predetermined polarization of the image light emitted from display region 1502 may be S polarization, and the reflection characteristics of polarization separator 101D and polarization separator 101E may have the P polarization and S polarization characteristics interchanged. In this case, the illustrated P polarization and S polarization are both reversed, but the optical design, such as the optical path, can be realized in exactly the same way.

[0247] By using the optical system of FIG. 16B described above, it is possible to realize a space floating image display device in which the distance in the depth direction between the two layers of space floating images is made closer.

[0248] Next, another example of the optical system and optical path of the space floating image display device that displays a plurality of layers of space floating images with different protrusion amounts from the optical system of the third embodiment will be described with reference to FIG. 17A.

[0249] Fig. 17A shows an example of the configuration and optical path of an optical system of a space-floating image display device that displays multiple layers of space-floating images. The optical system of Fig. 17A has only one display device 1 as the display device, which is the image source. In the example of Fig. 17A, two display areas, display area 1501 and display area 1502, are provided on the display screen of display device 1. The optical system of Fig. 17A displays space-floating image 3D corresponding to display area 1501. The optical system of Fig. 17A displays space-floating image 3E corresponding to display area 1502.

[0250] In the example of Fig. 17A, when a user views the space floating image 3E and the space floating image 3D from the direction of arrow A, the space floating image 3E appears to be displayed in front of the space floating image 3D. Since the space floating image 3E and the space floating image 3D appear to overlap from the user's perspective, these space floating images are viewed as a space floating image with two layers of depth.

[0251] Next, the detailed configuration of the optical system in Fig. 17A will be described. The configuration of the display device 1 is the same as in Example 1, so repeated description will be omitted. First, image light of a predetermined polarized light (P polarized light in the figure) is output from the display device 1. Image light of a predetermined polarized light (P polarized light in the figure) is output at both positions in the display area 1501 and the display area 1502.

[0252] Here, the image light of a predetermined polarization (P polarization in the figure) output from the display region 1501 travels as shown in the figure and enters the polarization separation member 101D. The polarization separation member 101D selectively transmits the predetermined polarization (P polarization in the figure) and reflects the other polarization (S polarization in the figure). Therefore, the image light of the predetermined polarization (P polarization in the figure) output from the display region 1501 passes through the polarization separation member 101D and reaches the retroreflector 2D to which the λ / 4 plate 21D is attached. The image light is retroreflected by the retroreflector 2D and travels again toward the polarization separation member 101D, whereby the image light has been converted from the predetermined polarization (P polarization in the figure) at the time of output from the display device 1 to the other polarization (S polarization in the figure) by passing through the λ / 4 plate 21D twice.

[0253] The image light that travels back toward the polarization separator 101D is the other polarized light (S-polarized light in the figure), and is therefore reflected by the polarization separator 101D toward the location where the user should be. The traveling direction of the image reflected by the polarization separator 101D is determined based on the angle at which the polarization separator 101D is placed. In the example of FIG. 17A, the image light that travels toward the polarization separator 101D is reflected at a right angle by the polarization separator 101D and travels as shown in the figure. The image light reflected by the polarization separator 101D forms a space-floating image 3D.

[0254] Next, the image light of a predetermined polarization (P-polarized light in the figure) emitted from the display region 1502 travels as shown in the figure and enters the polarization separation member 101E. The polarization separation member 101E selectively transmits the predetermined polarization (P-polarized light in the figure) and reflects the other polarization (S-polarized light in the figure). Therefore, the image light traveling toward the polarization separation member 101E passes through the polarization separation member 101E and travels toward the specular reflector 24 to which the λ / 4 plate 21F is attached. The image light that is specularly reflected by the specular reflector 24 and travels again toward the polarization separation member 101E has been converted from the predetermined polarization (P-polarized light in the figure) to the other polarization (S-polarized light in the figure) by passing through the λ / 4 plate 21F twice.

[0255] The image light that travels again toward the polarization separation member 101E is the other polarized light (S-polarized light in the figure), so it is reflected by the polarization separation member 101E and reaches the retroreflector 2E to which the λ / 4 plate 21E is attached. The traveling direction of the image reflected by the polarization separation member 101E is determined based on the angle at which the polarization separation member 101E is disposed. In the example of FIG. 17A, the image light that travels toward the polarization separation member 101E is reflected at a right angle by the polarization separation member 101E and travels as shown in the figure. The image light that is retroreflected by the retroreflector 2E and travels again toward the polarization separation member 101E has been converted from the other polarized light (S-polarized light in the figure) to the predetermined polarized light (P-polarized light in the figure) by passing through the λ / 4 plate 21E twice.

[0256] The image light that travels back toward the polarization separation member 101E is of a predetermined polarization (P polarization in the figure), and therefore passes through the polarization separation member 101E. As shown in the figure, the image light that has passed through the polarization separation member 101E travels toward the polarization separation member 101D. As described above, the polarization separation member 101D selectively transmits light of a predetermined polarization (P polarization in the figure) and reflects the other polarization (S polarization in the figure). Therefore, the image light from the retroreflector 2E, which is of a predetermined polarization (P polarization in the figure), passes through the polarization separation member 101D and travels toward the position where the user should be. The image light that has passed through the polarization separation member 101D forms the spatially floating image 3E.

[0257] In the example of Figure 17A, a light shielding plate is provided between the optical path of the image light output from display area 1501 and the optical path of the image light output from display area 1502 to prevent each image light from leaking into the optical path of the other image light.

[0258] 17A, polarization separating member 101D is disposed at an angle of 45 degrees relative to the traveling direction of the image light from display device 1. Polarization separating member 101E is disposed at an angle of 45 degrees relative to the traveling direction of the image light from display device 1 in a direction different from that of polarization separating member 101D.

[0259] As a result, the image light forming the space floating image 3E and the image light forming the space floating image 3D travel in the same direction toward the position where the user should be. In order to configure this, in the example of FIG. 17A, the user moves in the direction of arrow A (y direction) to the space floating image 3E When the user views the floating image 3E and the floating image 3D, the floating image 3E, the floating image 3D, the polarization separation member 101D, the polarization separation member 101E, and the retroreflector 2E are arranged on the same straight line as seen by the user (for example, in the example of Figure 17A, this is the straight line of the optical path from the retroreflector 2E to the floating image 3E, which extends in the direction of the user).

[0260] In addition, at this time, the display device 1, the retroreflector 2D, and the specular reflector 24 are arranged at positions that are not aligned on the same straight line. In the example of FIG. 17A, when the user views the space floating images 3E and 3D from the direction of the arrow A (y direction), the space floating images 3E and 3D are aligned. The positions of the polarization separation members 101D and 101E are determined so that the center of the left-right direction (x direction) of the floating image 3E coincides with the center of the left-right direction (x direction) of the floating image 3D.

[0261] From the user's perspective, the center of the horizontal direction (x direction) of the image of the floating image 3E and the center of the floating image 3F are If the center of the 3D image in the left-right direction (x direction) is aligned with the center of the 3D image, it will be more clear to the user. This is more suitable because it is easier to view and there is no need for video content producers to consider offsets. It also simplifies the optical layout, which is more suitable.

[0262] 17A, the characteristics of P polarization and S polarization may be interchanged. Specifically, the predetermined polarization of the image light emitted from display region 1501 of display device 1 may be S polarization, the predetermined polarization of the image light emitted from display region 1502 of display device 1 may be S polarization, and the reflection characteristics of polarization separator 101D and polarization separator 101E may have the P polarization and S polarization characteristics interchanged. In this case, the illustrated P polarization and S polarization are both reversed, but the optical design, such as the optical path, can be realized in exactly the same way.

[0263] According to the optical system of FIG. 17A described above, an optical system that forms a floating image with two layers of depth can be realized using a single display device. Note that a configuration may also be adopted in which separate display devices are provided for display area 1501 and display area 1502. However, a configuration that includes multiple display devices will require an increased number of corresponding circuits, which may result in relatively high costs. Therefore, if a floating image with two layers of depth is formed using only a single display device as in FIG. 17A, an optical system that forms a floating image with two layers of depth can be realized at lower cost.

[0264] 17A, the optical path length of the image light emitted from the display area 1501 of the display device 1 to form the space floating image 3D, and the optical path length of the image light emitted from the display area 1502 of the display device 1 to form the space floating image 3E will be explained. These optical path lengths will be explained using the optical path length of the light ray emitted from the center of the display area 1501 in the normal direction, and the optical path length of the light ray emitted from the center of the display area 1502 in the normal direction. The same applies to the following explanation.

[0265] 17A, the optical path length of the image light emitted from display region 1501 of display device 1 to reach polarization separation member 101D is equal to the optical path length of the image light emitted from display region 1502 of display device 1 to reach polarization separation member 101E. This is similar to the optical systems of FIGS. 15A and 15B.

[0266] Here, in the optical system of Figure 15A and the optical system of Figure 15B, in order to ensure that the spatially floating image 3E formed by the image light emitted from the display area 1502 of the display device 1 is sufficiently projected from the optical system, it is necessary to increase the optical path length from the image light emitted from the display area 1502 of the display device 1 to form the spatially floating image 3E, and as a result, it is necessary to ensure a relatively long distance from the display device 1 to the polarization separation member 101D and the polarization separation member 101E.

[0267] 17A, the angle of the polarization separator 101E is arranged to be 90 degrees different from the angle of the polarization separator 101D, and an optical path that goes back and forth between the polarization separator 101E and the specular reflector 24 is added to the optical path of the image light emitted from the display area 1502 of the display device 1 to form the space-floating image 3E. As a result, in the optical system of Fig. 17A, the optical path length of the image light emitted from the display area 1502 of the display device 1 to form the space-floating image 3E is longer than the optical path length of the image light emitted from the display area 1502 of the display device 1 to form the space-floating image 3E in the optical systems of Fig. 15A and Fig. 15B.

[0268] Therefore, in the optical system of Figure 17A, even if the distance from the display device 1 to the polarization separation member 101D and the polarization separation member 101E is shorter than in the optical systems of Figures 15A and 15B, it is possible to ensure a sufficient amount of projection from the optical system for the spatially floating image 3E formed by the image light emitted from the display area 1502 of the display device 1.

[0269] 17A, the optical path length of the image light emitted from the display area 1502 of the display device 1 until it forms the space-floating image 3E can be changed depending on the distance D between the specular reflector 24 and the display surface of the display device 1. Therefore, in the space-floating image display device, the position of the specular reflector 24 can be determined so that the desired amount of protrusion of the space-floating image 3E is achieved. Furthermore, the distance between the space-floating image 3E and the space-floating image 3D will also change depending on this distance D. Therefore, in a space-floating image display device that displays multiple layers of space-floating images with different protrusion amounts from the optical system, the position of the specular reflector 24 can be determined so that the distance between the multiple layers of space-floating images is the distance required for the product.

[0270] Here, as can be seen from a comparison with the optical systems of FIG. 15A and FIG. 15B, in the optical system of FIG. 17A, the distance from the display device 1 to the polarization separation members 101D and 101E can be made relatively short, and therefore the volume of the optical system is smaller. Furthermore, in the optical system of FIG. 17A, depending on the position of the specular reflector 24, the distance between the space-floating images 3E and 3D can be set shorter than in the optical systems of FIG. 15A and FIG. 15B. In other words, the optical system of FIG. 17A can realize a more compact optical system for a space-floating image display device that displays multiple layers of space-floating images. Furthermore, the distance between the multiple layers of space-floating images can be set to a desired position with a simple configuration, which is more preferable.

[0271] In addition, when the optical system of Fig. 17A is incorporated into a space-floating image display device, it can be realized by replacing the optical system in the space-floating image display device described in Example 1 with the optical system of Fig. 17A. Specifically, the optical system of Fig. 17A may be replaced with the optical system of the space-floating image display device of Fig. 4E, Fig. 4F, Fig. 4G, Fig. 4K, or Fig. 4L.

[0272] In this case, a space-floating image display device that forms a space-floating image with two layers of depth can be realized in each figure. In particular, in Figures 4K and 4L, a space-floating image with two layers of depth can be formed on the front side as seen from the user of the transmissive self-luminous image display device 1650. In this case, it is possible to form an image of a space-floating image with two layers of depth and an image of three layers of different depths on the transmissive self-luminous image display device 1650 so that the user can see it.

[0273] Next, an example of a space-floating image display device 1000 equipped with the optical system of Fig. 17A will be described with reference to Fig. 17B. Fig. 17B shows an example of the configuration of a space-floating image display device 1000 that displays multiple layers of space-floating images with different protrusion amounts. The space-floating image display device 1000 shown in the figure incorporates the optical system described in Fig. 17A. In the example of Fig. 17B, the optical system is oriented so that the display device 1 and the specular reflector 24 face each other in the left-right direction (x direction) of the user. 17A. In Fig. 17B, the symbols of other elements in the optical system of Fig. 17A are omitted. As shown in Fig. 17B, the space-floating image display device 1000 can display two-layered space-floating images, space-floating image 3E and space-floating image 3D, toward the user 230. In addition, since the optical system of Fig. 17A itself is relatively small, the space-floating image display device 1000 can also be realized in a relatively small size.

[0274] According to the space-floating image display device 1000 of FIG. 17B explained above, a space-floating image display device that displays a plurality of layers of space-floating images with different protrusion amounts can be realized in a smaller size.

[0275] Next, another example of the space floating image display device 1000 including the optical system of FIG. 17A will be described with reference to FIG. 17C.

[0276] Fig. 17C shows an example of the configuration of a space-floating image display device 1000 that displays multiple layers of space-floating images with different projection amounts. The optical system described in Fig. 17A is incorporated into the space-floating image display device 1000 shown in the figure. The example in Fig. 17C is an example in which the optical system is arranged so that the display device 1 and the specular reflector 24 face each other in the vertical direction (z direction) as seen from the user. In Fig. 17C, the symbols of other elements in the optical system in Fig. 17A are omitted.

[0277] As shown in FIG. 17C , the space-floating image display device 1000 can display two-layer space-floating images, namely, space-floating image 3E and space-floating image 3D, toward the user 230. Here, in the example of FIG. 17C , an aerial operation detection sensor may be provided so that user operations on each of the space-floating image 3E and the space-floating image 3D can be detected. Specifically, as shown in FIG. 17C , an aerial operation detection sensor 1351D is provided for detecting user operations on the space-floating image 3D. In addition, an aerial operation detection sensor 1351E is provided for detecting user operations on the space-floating image 3E. For example, the aerial operation detection sensor 1351 in FIG. 3 can be replaced with these two sensors. The aerial operation detection unit 1350 in FIG. 3 can determine whether or not a user operation has been performed on each of the space-floating image 3E and the space-floating image 3D based on signals from these sensors.

[0278] Here, when user 230 attempts to operate, with a finger, floating in space image 3E displayed in front of the user, the user's finger will touch floating in space image 3E, but it is not necessary for the user's finger to touch floating in space image 3D. Therefore, if an operation input signal is not detected by mid-air operation detection sensor 1351D for detecting user operations on floating in space image 3D, but an operation input signal is detected by mid-air operation detection sensor 1351E for detecting user operations on floating in space image 3E, mid-air operation detection unit 1350 will determine that "the user is performing a user operation on floating in space image 3E."

[0279] In contrast, when user 230 tries to use his / her fingers to operate the floating image 3D, which is displayed at the back from the user's perspective, the user's fingers will touch the floating image 3D, but since the floating image 3E is in front, there is a high possibility that the user's fingers or arms will touch the floating image 3E.

[0280] Therefore, even if an operation input signal is detected by the aerial operation detection sensor 1351E, if the operation input signal is detected by the aerial operation detection sensor 1351D for detecting user operations on the floating-in-space image 3D, the aerial operation detection unit 1350 may determine that "the user is performing a user operation on the floating-in-space image 3D." In this case, the operation input signal detected by the aerial operation detection sensor 1351E may be configured to be ignored.

[0281] It is also possible to configure the operated icon to be displayed simultaneously on two layers of optical images with different depths by shifting the xz direction position of the operated icon displayed on the space floating image 3E and the xz direction position of the operated icon displayed on the space floating image 3D. In such a case, it is not necessarily necessary to perform the process of ignoring the operation input signal detected by the above-mentioned aerial operation detection sensor 1351E.

[0282] As explained above, according to the space-floating image display device 1000 of Fig. 17C, it is possible to realize a more compact space-floating image display device that displays multiple layers of space-floating images with different protrusion amounts. Also, according to the space-floating image display device 1000 of Fig. 17C, it is possible to more suitably detect user operations on each of the multiple layers of different space-floating images.

[0283] Next, another example of the space floating image display device 1000 including the optical system of FIG. 17A will be described with reference to FIG. 17D.

[0284] FIG. 17D shows an example of the configuration of a space-floating image display device 1000 that displays multiple layers of space-floating images with different projection amounts. The optical system described in FIG. 17A is incorporated into the space-floating image display device 1000 shown in the figure. In the example of FIG. 17D, the display device 1 and the specular reflector 24 face each other in the depth direction (y direction) as seen from the user, and the optical system is arranged so that the space-floating images project at an angle toward the user from the vertical z direction. In FIG. 17D, the symbols for other elements in the optical system of FIG. 17A are omitted. As shown in FIG. 17D, the space-floating image display device 1000 can display two layers of space-floating images, a space-floating image 3E and a space-floating image 3D, toward the user 230.

[0285] According to the space-floating image display device 1000 of FIG. 17D, it is possible to realize a space-floating image display device that displays space-floating images of multiple layers with different protruding amounts, which the user can look into from above.

[0286] Next, an example of a display example in a space-floating image display device that displays multiple layers of space-floating images with different amounts of protrusion will be described using Fig. 18A. In Fig. 18A, space-floating image 3-1 and space-floating image 3-2 are multiple layers of space-floating images with different amounts of protrusion. To simplify the explanation, the hardware of the space-floating image display device itself will be omitted.

[0287] Space-floating image 3-1 is displayed closer to the user than space-floating image 3-2. Space-floating image 3-2 is displayed further back than space-floating image 3-1. Display object 1810 is an object displayed in the display image of space-floating image 3-1. Display object 1821 is an object displayed in the display image of space-floating image 3-2. For example, if the space-floating image display device performing the display of FIG. 18A is a space-floating image display device arranged as shown in FIG. 17D, multiple layers with different protrusion amounts will be displayed in a direction close to the vertical direction. Therefore, in the display example of FIG. 18A, display object 1821 of space-floating image 3-2, which is displayed further back than space-floating image 3-1, displays a virtual shadow 1822 that appears to be caused by display object 1810 of space-floating image 3-1, which is displayed closer to the user than space-floating image 3-2. Here, the virtual shadow may be displayed by displaying the corresponding part in black, or by reducing the brightness of the video signal of the corresponding part. Also, the saturation of the video signal of the corresponding portion may be reduced. These processes may be performed by the video control unit 1160 in FIG.

[0288] As shown in FIG. 18A, by displaying the virtual shadow of an object displayed in a space-floating image on an object displayed in another space-floating image at a different depth, the depth relationship between the two space-floating images, both of which are space-floating images, becomes easier to visually recognize, and the sense of reality of the space-floating images felt by the user can be more appropriately improved.

[0289] Furthermore, as shown in FIG. 18A , when display objects are displayed in a plurality of space-floating images having different depths from the user's perspective, the brightness of display object 1810 of space-floating image 3-1, which is displayed in front of the user, may be configured to be brighter than display object 1821 of space-floating image 3-2, which is displayed behind the user's perspective. This may be achieved by changing the brightness optically, or by changing the brightness through video signal processing. By displaying in this manner, even if a display object in front of the user and a display object behind the user overlap, the display object in the front, which appears bright, becomes easier for the user to recognize, and the display object behind the user, which appears dark, becomes harder for the user to recognize, thereby generating a pseudo-occlusion. By making the user recognize such pseudo-occlusion between objects in a plurality of space-floating images having different depths, the sense of reality of the space-floating images experienced by the user can be more appropriately improved.

[0290] The display example of FIG. 18A explained above can be used in a space floating image display device including any of the optical systems shown in FIGS. 15A to 17A, for example.

[0291] According to the display example of FIG. 18A explained above, in a space-floating image display device that displays a plurality of layers of space-floating images with different amounts of protrusion, the sense of reality of the space-floating image felt by the user can be more suitably improved.

[0292] Next, an example of a display in a space-floating image display device that displays multiple layers of space-floating images with different projection amounts will be described with reference to FIG. 18B. Space-floating image 3-1 and space-floating image 3-2 shown in FIG. 18B are multiple layers of space-floating images with different projection amounts. Space-floating image 3-1 is displayed in front of the user, and space-floating image 3-2 is displayed behind the user. The display example of FIG. 18B can be used, for example, in a space-floating image display device equipped with any of the optical systems shown in FIGS. 15A to 17A. The display example of FIG. 18B can be used, for example, in any of the space-floating image display devices shown in FIGS. 17B to 17D. For example, when the display example of FIG. 18B is applied to the space-floating image display device 1000 of FIG. 17B, space-floating image 3-1 corresponds to space-floating image 3E of FIG. 17B, and space-floating image 3-2 corresponds to space-floating image 3D of FIG. 17B.

[0293] In Fig. 18B, display object 1850 is an object displayed in the display image of space floating image 3-1. In the example of Fig. 18B, display object 1850 is a display object of a character. In the example of Fig. 18B, the character is a human character. Display object 1855 and display object 1856 are objects displayed in the display image of space floating image 3-2. In the example of Fig. 18B, display object 1855 and display object 1856 are background objects. In the example of Fig. 18B, the background is a pillar.

[0294] 18B, a character display object is placed near the center in the horizontal direction of space floating image 3-1, which is the foreground, and display object 1855 and display object 1856, which are background objects, are placed on the left and right sides of space floating image 3-2, which is the background, avoiding the center. In this display example, the main content of the display content (the content that is desired to draw the user's attention) is display object 1850, which is the character display object. Display object 1855 and display object 1856 are secondary content, and are displayed to more preferably allow the user to recognize display object 1850, which is the main content.

[0295] The advantages of such an object display layout will be explained using a space-floating image display device that displays space-floating images in multiple layers with different protrusion amounts. For example, in the case of a space-floating image display device that displays space-floating images in a single layer, when a character display object is displayed in the space-floating image, there are no objects in front of or behind the character display object that serve as a reference for depth, so it may be difficult for the user to recognize the depth of the display position of the space-floating image. In contrast, in the display example of FIG. 18B , a space-floating image display device that displays space-floating images in multiple layers with different protrusion amounts is used to display a character display object 1850 near the center in the horizontal direction of space-floating image 3-1, and in the background space-floating image 3-2, pillar display object 1855 and pillar display object 1856 are placed on the left and right positions.

[0296] At this time, because there is a difference in depth between the spatially floating images 3-1 and 3-2, when the user moves their head to change the viewpoint, the principle of motion parallax causes the horizontal distance and relative position of the pillar display object 1855 and pillar display object 1856 to change relative to the character display object displayed in the horizontal center. This allows the user to more clearly recognize that the character display object displayed in the horizontal center of the spatially floating image 3-1 is located closer to the user than the pillar display object 1855 and pillar display object 1856 displayed on the left and right of the spatially floating image 3-2. Here, the two floating images in the air, the spatially floating image 3-1 and the spatially floating image 3-2, are perceived as overlapping depending on the relationship between their respective display ranges. In this case, if the image of the floating image 3-2 at the back from the user's perspective is bright and the image of the floating image 3-1 at the front from the user's perspective is dark, the displayed image of the floating image 3-2 at the back from the user's perspective will be visible through the displayed image of the floating image 3-1 at the front from the user's perspective, which may cause the user to have an incorrect perception of the front and back of the displayed object.

[0297] Therefore, image processing may be performed in which the brightness of the display object area of ​​the image of the space-floating image 3-1, which is closer to the user, is adjusted to be brighter overall, and the brightness of the display object area of ​​the image of the space-floating image 3-2, which is farther from the user, is adjusted to be darker overall. However, depending on the character design of the display object displayed in the space-floating image 3-1, it may not be possible to brighten the overall brightness of the display object area. For example, this may be the case when the character's costume is dark gray. In the case of a character with such a dark design, it is desirable that the display object of the space-floating image 3-2, which is farther from the user, does not overlap with the display object of the character in the space-floating image 3-1, which is closer to the user.

[0298] Therefore, as shown in FIG. 18B, the character display object is placed near the center in the horizontal direction of the foreground, Floating in Space Image 3-1, and in the background, Floating in Space Image 3-2, display objects 1855 and 1856, which are background objects, are placed on the left and right sides of the foreground, avoiding the center in the horizontal direction where the character display object is located in Floating in Space Image 3-1. This makes it possible to display characters with various character designs so that they do not overlap with the background objects when viewed from the user, and the effect of motion parallax described above makes it possible to more clearly recognize the display position in the depth direction where character display object 1850, the main content, is displayed, while maintaining the user's depth perception more favorably.

[0299] Note that the motion parallax in the display example of FIG. 18B is motion parallax that occurs based on the actual spatial positions of the real images, Floating in Space Image 3-1 and Floating in Space Image 3-2, and is not pseudo-motion parallax. This is different from a technology that generates pseudo-motion parallax by image processing based on the user's viewpoint position. The technology shown in the display example of FIG. 18B does not require image processing based on the user's viewpoint position, and the amount of processing can be relatively reduced. Furthermore, while technologies that require image processing based on the user's viewpoint position often do not easily support simultaneous viewing by multiple people, the technology shown in the display example of FIG. 18B does not require image processing based on the user's viewpoint position, so it is possible to obtain a more suitable motion parallax effect even when multiple different users view from different angles.

[0300] Next, using Fig. 18C, an example of the display image of display device 1, which is the image that is the basis for space-floating image 3-1 and space-floating image 3-2 explained in Fig. 18B, will be explained. Fig. 18C is an example of the display image of display device 1 when the display example of Fig. 18B is displayed using space-floating image display device 1000 of Fig. 17B equipped with the optical system of Fig. 17A. Display screen 1801 of display device 1 includes display area 1501 and display area 1502. In space-floating image display device 1000 of Fig. 17B, the display image of display area 1502 of display device 1 is displayed in the air as space-floating image 3E, which corresponds to space-floating image 3-1 of Fig. 18B.

[0301] In the space-floating image display device 1000 of Fig. 17B, the display image of display area 1501 of display device 1 is displayed in the air as space-floating image 3D, which corresponds to space-floating image 3-2 of Fig. 18B. The display image of display area 1502 displayed in space-floating image 3-1 of Fig. 18B is displayed closer to the user than the display image of display area 1501 displayed in space-floating image 3-2 of Fig. 18B. Note that when the optical system of Figs. 15A to 16B is used, the front-to-back relationship in the depth direction from the user's perspective of space-floating image 3D and space-floating image 3E is reversed, so the display image of display area 1501 is displayed closer to the user than the display image of display area 1502 in the air.

[0302] As shown in FIG. 18C, the space floating image display device 1000 according to this embodiment can display two image sources, each displayed at a different depth in the air, on a single piece of hardware, the display device 1. Images are displayed in display area 1501 and display area 1502 of the display device 1, respectively, and the images of each frame of the two images are stored as part of a single image in the frame memory of the display device 1. Therefore, compared to a configuration in which two images are displayed using different display devices, there is no need to set up a complex synchronization system or the like to synchronize the two images, which is more preferable. Compared to a configuration in which two images are displayed using different display devices, there is no need to set up two systems of hardware required for various processes, such as display memory, and this can be realized more inexpensively.

[0303] 18C, a gap 1807 is provided between display region 1501 and display region 1502. In the region of gap 1807, display device 1 fixes the image to black display. The reason for this will be explained below. In any of the optical systems shown in FIGS. 15A to 17A, as described above, a light-shielding plate is provided between display region 1501 and display region 1502 on the emission surface of the display screen of display device 1. The light-shielding plate is provided to prevent, as much as possible, the image light emitted from display region 1501 and the image light emitted from display region 1502 from mixing with each other in their optical paths.

[0304] Furthermore, it is desirable to provide a gap 1807 between display region 1501 and display region 1502, and to make the width of gap 1807 larger than the thickness of a light shielding plate provided between display region 1501 and display region 1502 on the emission surface of the display screen of display device 1. This makes it possible to prevent the image light emitted from display region 1501 and the image light emitted from display region 1502 from being vignetted by the light shielding plate, and to prevent mixing of each other in their optical paths as much as possible. Note that although the image in the region of gap 1807 has been described as being fixed to a black display, it may also be expressed as a content non-display region where no content is displayed.

[0305] Here, a first processing example of the process for realizing the display example of Fig. 18C will be described using the configuration of the space floating image display device 1000 of Fig. 3. The first processing example is an example in which the image to be displayed in the display area 1501 and the image to be displayed in the display area 1502 are respectively reproduced from the storage unit 1170 and displayed.

[0306] Specifically, content having video information of the character of display object 1850 and background video information including display objects 1855 and 1856, which are background objects, is stored in storage unit 1170, and video control unit 1160 plays back the video information of the character of display object 1850 and places the played back video information at a position corresponding to display area 1502 of display device 1 in Fig. 18C. Video control unit 1160 further plays back background video information including display objects 1855 and 1856 stored in storage unit 1170 and controls display in display area 1501 of display device 1 in Fig. 18C.

[0307] Furthermore, a second processing example for the process for realizing the display example of Fig. 18C will be described using the configuration of the space floating image display device 1000 of Fig. 3. The second processing example is an example in which a content creator who understands in advance the layout of the entire screen of the display screen 1801 of the display device 1 shown in Fig. 18C and the display areas 1501 and 1502 creates video content corresponding to the display screen 1801 including the display areas 1501 and 1502, stores the content in the storage unit 1170, and the video control unit 1160 plays back the video of the content and controls it to be displayed on the entire screen of the display screen 1801 of the display device 1.

[0308] The video of the video content has content corresponding to display screen 1801, and includes a video of display object 1850, which is a character, at a position corresponding to display area 1502, and includes videos of display objects 1855 and 1856, which are background objects, at a position corresponding to display area 1501. Since the video of the content stored in storage unit 1170 corresponds to the layout of the entire screen of display screen 1801 in FIG. 18C and display areas 1501 and 1502, video control unit 1160 only needs to play back the video of the content and perform control so that it is displayed across the entire screen of display screen 1801 of display device 1. This eliminates the need for complex image superimposition processing when displaying the content, thereby reducing the amount of processing.

[0309] Furthermore, a third processing example for the process of realizing the display example of Fig. 18C will be described using the configuration of the space-floating image display device 1000 of Fig. 3. In the third processing example, a content creator who understands in advance the layout of the entire screen of the display screen 1801 of the display device 1 shown in Fig. 18C and the display area 1501 and the display area 1502 creates video content corresponding to the display screen 1801 including the display area 1501 and the display area 1502, and stores the video content in an external device different from the space-floating image display device 1000.

[0310] The external device and the space-floating image display device 1000 are connected in advance so that a video output signal from the external device can be input from the video signal input unit 1131 of the space-floating image display device 1000 in Fig. 3. The external device outputs a video signal of video content corresponding to the display screen 1801 including the display area 1501 and the display area 1502, and inputs it to the video signal input unit 1131 of the space-floating image display device 1000. The video control unit 1160 plays the video signal of the video content input to the video signal input unit 1131, and controls the display device 1 to display it on the display screen 1801.

[0311] The details of the video content are the same as those in the second processing example for realizing the display example of Fig. 18C, and therefore a repeated explanation will be omitted. At the time when the video of the content is input to video signal input unit 1131, it corresponds to the layout of the entire screen of display screen 1801 of Fig. 18C and display area 1501 and display area 1502. Therefore, video control unit 1160 only needs to play back the video of the content and control it to be displayed on the entire screen of display screen 1801 of display device 1. When displaying the content, it is not necessarily necessary to perform complex image superimposition processing, and the amount of processing can be reduced.

[0312] A fourth processing example for realizing the display example of Fig. 18C will be described using the configuration of the space floating image display device 1000 of Fig. 3. The fourth processing example is an example in which an image to be displayed in the display area 1501 and an image to be displayed in the display area 1502 are generated by rendering from a 3D model using an image generation program.

[0313] Specifically, first, an image generation program capable of generating a rendered image of a 3D model of a character corresponding to display object 1850 and generating a rendered image of 3D models of background objects corresponding to display objects 1855 and 1856 is stored in storage unit 1170. Control unit 1110 reads out the image generation program from storage unit 1170 and loads it in memory 1109. Control unit 1110 executes the image generation program loaded in memory 1109, and the image generation program renders the 3D model of the character to generate an image of display object 1850.

[0314] The image control unit 1160 performs control to display the image of the generated display object 1850 in the display area 1502 of Fig. 18C. In parallel, the image generation program renders a 3D model of the background object to generate images of the display objects 1855 and 1856. The image control unit 1160 then performs control to display the images of the generated display objects 1855 and 1856 in the display area 1501 of Fig. 18C.

[0315] In the example of FIG. 18B, the character that is the main content is a human character, but it may also be an animal character or a robot character. It may also be a character related to a so-called avatar used in a virtual space. Here, the display object 1850 may be a character image rendered from a 3D model. Alternatively, a 2D animation character may be used. Alternatively, the character image may be a live-action image of a person or the like.

[0316] 18B, display object 1855 and display object 1856, which are the secondary content, are exemplified as objects representing pillars, but they may also be virtual frame objects, or furniture or equipment objects placed in a space where the character, which is the primary content, is set to exist. Any background object located behind the character in the space where the character is set to exist may be used.

[0317] According to the display examples of Figs. 18B and 18C explained above, in a space floating image display device that displays multiple layers of space floating images with different amounts of protrusion, it is possible to realize a display that allows the user to more clearly recognize the display position in the depth direction of display objects such as characters, which are the main content, while more appropriately maintaining the user's perception of depth.

[0318] Next, using Fig. 18D, we will explain another example of a display example in a space-floating image display device that displays multiple layers of space-floating images with different amounts of protrusion. Space-floating image 3-1 and space-floating image 3-2 shown in Fig. 18D are multiple layers of space-floating images with different amounts of protrusion. Space-floating image 3-1 is displayed in the foreground as seen from the user, and space-floating image 3-2 is displayed in the background as seen from the user.

[0319] The display example of Fig. 18D can be used, for example, in a space-floating image display device equipped with any of the optical systems shown in Fig. 15A to Fig. 17A. The display example of Fig. 18D can be used, for example, in any of the space-floating image display devices shown in Fig. 17B to Fig. 17D. For example, when the display example of Fig. 18D is applied to the space-floating image display device 1000 of Fig. 17B, the space-floating image 3-1 corresponds to the space-floating image 3E of Fig. 17B, and the space-floating image 3-2 corresponds to the space-floating image 3D of Fig. 17B.

[0320] In FIG. 18D, display object 1851 is an object displayed in the display image of space floating image 3-2. In the example of FIG. 18D, display object 1851 is a display object of a character. In the example of FIG. 18D, the character is a human character. In the example of FIG. 18D, display object 1857 and display object 1858 are objects displayed in the display image of space floating image 3-1. In the example of FIG. 18D, display object 1857 and display object 1858 are foreground objects. In the example of FIG. 18D, a foreground object is an object that should be displayed spatially in front of (closer to the user than) the main content of the display content. In the example of FIG. 18D, the background is text.

[0321] 18D, text is displayed as secondary content in the foreground of display object 1851, which is the main content of the display content and is a character. This display example is, for example, an animated video of the character of display object 1851 singing a song, and the lyrics of the song are displayed in the foreground in display objects 1857 and 1858. In this figure, display object 1857 is an example of horizontally written text, and display object 1858 is an example of vertically written text.

[0322] In the example of Fig. 18D, display objects 1857 and 1858, which are the lyrics of the song being sung by the character of display object 1851 displayed in the display image of floating in space image 3-2, are displayed in synchronization with the animation of the character singing in floating in space image 3-1 in the foreground. In other words, as the singing progresses, the lyrics to be displayed can be changed in sync with the singing. There are various ways to express the display position, character direction, size, font, etc. of the lyrics. The characters may be scrolled in accordance with the progress of the singing, or the characters may be switched and displayed a few characters at a time while changing their position and size.

[0323] Unlike the display example in Fig. 18B, there is no particular need to avoid overlapping between character display object 1851 and letter display object 1857 or display object 1858. If this is effective as a display effect, they may be displayed so that they appear to overlap when viewed from the front, as in Fig. 18D.

[0324] Generally, there is technology for superimposing video and text on flat 2D displays. However, when video and text are superimposed on a flat 2D display without any special processing, motion parallax does not occur in the relative positions of the video and text, even when the user changes their viewpoint. Therefore, even if the text is superimposed on top of the video, the user perceives it as if it is superimposed on the same plane as the video, and it is not easy for the user to recognize that the text is displayed at a different depth from the video.

[0325] Furthermore, in this case, when a user changes their viewpoint on a flat 2D display, the image on which text information is superimposed is likely to be perceived as a flat image because no motion parallax occurs in the relative positions of the image and the text information, making it difficult for the user to view the image in three dimensions.

[0326] 18D, since the space floating image 3-1 and the space floating image 3-2 are real images with a difference in depth, when the user moves their head to change the viewpoint, the principle of motion parallax causes the distance and relative position of the character display object 1857 and display object 1858 to change with respect to the position of the character display object 1851. This allows the user to easily recognize that the character display object 1857 and display object 1858 are displayed in front of the character display object 1851.

[0327] At this time, it is easy to recognize that the character display object 1857 and display object 1858 displayed in space floating image 3-2 are not on the same plane as the character display object 1851 displayed in space floating image 3-1. This is more preferable because the display of character display object 1851 displayed in space floating image 3-1 is not restricted by planar recognition, such as being on the same plane as the character display object 1857 and display object 1858.

[0328] Note that the motion parallax in the display example of FIG. 18D is motion parallax generated based on the actual spatial positions of the real images, Floating in Space Image 3-1 and Floating in Space Image 3-2, and is not pseudo-motion parallax. This is different from a technology that generates pseudo-motion parallax by image processing based on the user's viewpoint position. The technology shown in the display example of FIG. 18D does not require image processing based on the user's viewpoint position, and the amount of processing can be relatively reduced. Furthermore, while technologies that require image processing based on the user's viewpoint position often do not easily support simultaneous viewing by multiple people, the technology shown in the display example of FIG. 18D does not require image processing based on the user's viewpoint position, so it is possible to obtain a more suitable motion parallax effect even when multiple different users view from different angles.

[0329] 18D shows an example in which display object 1857 and display object 1858 are displayed simultaneously, but these are examples of character display modes and do not necessarily need to be displayed simultaneously. There may be times when neither is displayed.

[0330] Next, using Fig. 18E, we will explain an example of the display image of the display device 1, which is the image that is the source of the space-floating image 3-1 and the space-floating image 3-2 explained in Fig. 18D. Fig. 18E is an example of the display image of the display device 1 when displaying the display example of Fig. 18D using the space-floating image display device 1000 of Fig. 17B equipped with the optical system of Fig. 17A.

[0331] Display screen 1801 of display device 1 includes display area 1501 and display area 1502. In space-floating image display device 1000 of Fig. 17B, the display image in display area 1502 of display device 1 is displayed in the air as space-floating image 3E, which corresponds to space-floating image 3-1 of Fig. 18D. In space-floating image display device 1000 of Fig. 17B, the display image in display area 1501 of display device 1 is displayed in the air as space-floating image 3D, which corresponds to space-floating image 3-2 of Fig. 18D. The display image in display area 1502 displayed in space-floating image 3-1 of Fig. 18D is displayed closer to the user than the display image in display area 1501 displayed in space-floating image 3-2 of Fig. 18D.

[0332] When the optical system of Figures 15A to 16B is used, the front-to-back relationship in the depth direction from the user's perspective of the space floating image 3D and the space floating image 3E is reversed, so the display image of display area 1501 is displayed in front of the display image of display area 1502 in the air. Here, as in the example of Fig. 18E, the advantage of displaying two video sources that are displayed at different depth positions in the air on a single piece of hardware, display device 1, is the same as the advantage explained in Fig. 18C, so a repeated explanation will be omitted. Also, as in the example of Fig. 18E, the advantage of providing a gap 1807 area between display area 1501 and display area 1502 is the same as the advantage explained in Fig. 18C, so a repeated explanation will be omitted.

[0333] Here, a first processing example of the process for realizing the display example of Fig. 18E will be described using the configuration of the space floating image display device 1000 of Fig. 3. The first processing example is an example in which the image to be displayed in the display area 1501 and the image to be displayed in the display area 1502 are respectively reproduced from the storage unit 1170 and displayed.

[0334] Content having video information of the character of display object 1851 singing, text information of lyrics, and additional information such as timing information for displaying the text information is stored in storage unit 1170, and video control unit 1160 plays the video information of the character of display object 1851 singing and places the played video information in a position corresponding to display area 1501 of display device 1 in Figure 18E.

[0335] Video control unit 1160 may further play back the text information and additional information of the content stored in storage unit 1170, and use the display timing information to control the display in display area 1502 of display device 1 in Fig. 18E in synchronization with the display of the above-mentioned video information. At this time, if the additional information includes information such as the display position, size, font, and display color of the text information, the display position, size, font, and display color of the text information may be determined and displayed based on this information.

[0336] Also, a second processing example for the process for realizing the display example of Fig. 18E will be described using the configuration of the space floating image display device 1000 of Fig. 3. The second processing example is an example in which a content creator who understands in advance the layout of the entire screen of the display screen 1801 of the display device 1 shown in Fig. 18E and the display areas 1501 and 1502 creates video content corresponding to the display screen 1801 including the display areas 1501 and 1502, stores the content in the storage unit 1170, and the video control unit 1160 plays back the video of the content and controls it to be displayed on the entire screen of the display screen 1801 of the display device 1.

[0337] The video of the video content corresponds to display screen 1801, and includes a video of display object 1851, which is a singing character, at a position corresponding to display area 1501, and includes video of display objects 1857 and 1858, which are lyric letter objects, at a position corresponding to display area 1502. Since the video of the content stored in storage unit 1170 corresponds to the layout of the entire screen of display screen 1801 in FIG. 18E and display areas 1501 and 1502, video control unit 1160 only needs to play back the video of the content and perform control so that it is displayed across the entire screen of display screen 1801 of display device 1. This eliminates the need for complex image superimposition processing when displaying the content, thereby reducing the amount of processing.

[0338] Furthermore, a third processing example for the process of realizing the display example of Fig. 18E will be described using the configuration of the space-floating image display device 1000 of Fig. 3. In the third processing example, a content creator who understands in advance the layout of the entire screen of the display screen 1801 of the display device 1 shown in Fig. 18E and the display area 1501 and the display area 1502 creates video content corresponding to the display screen 1801 including the display area 1501 and the display area 1502, and stores the video content in an external device different from the space-floating image display device 1000.

[0339] The external device and the space-floating image display device 1000 are connected in advance so that a video output signal from the external device can be input from the video signal input unit 1131 of the space-floating image display device 1000 in Fig. 3. The external device outputs a video signal of video content corresponding to the display screen 1801 including the display area 1501 and the display area 1502, and inputs it to the video signal input unit 1131 of the space-floating image display device 1000. The video control unit 1160 plays the video signal of the video content input to the video signal input unit 1131, and controls the display device 1 to display it on the display screen 1801.

[0340] The details of the video content are the same as those in the second processing example for realizing the display example in Fig. 18E, and therefore a repeated explanation will be omitted. At the time when the video of the content is input to video signal input unit 1131, it corresponds to the layout of the entire screen of display screen 1801 in Fig. 18E and display areas 1501 and 1502. Therefore, video control unit 1160 only needs to play back the video of the content and control it to be displayed on the entire screen of display screen 1801 of display device 1. When displaying the content, it is not necessarily necessary to perform complex image superimposition processing, and the amount of processing can be reduced.

[0341] Also, a fourth processing example for realizing the display example of Fig. 18E will be described using the configuration of the space floating image display device 1000 of Fig. 3. The fourth processing example is an example in which an image to be displayed in the display area 1501 and an image to be displayed in the display area 1502 are generated by rendering from a 3D model using an image generation program.

[0342] Specifically, first, an image generation program capable of generating a rendered image of a 3D model of a character corresponding to display object 1851 and a rendered image of a model in 3D space of character information corresponding to display objects 1857 and 1858 is stored in storage unit 1170. Control unit 1110 reads out the image generation program from storage unit 1170 and loads it in memory 1109. Control unit 1110 executes the image generation program loaded in memory 1109, and the image generation program renders a 3D model of a character performing a singing animation to generate an image of display object 1851.

[0343] The video control unit 1160 performs control to display the video of the generated display object 1851 in the display area 1501 of FIG. 18E. In parallel, the video generation program renders a model of the character information in the 3D space to generate the video of the display object 1857 and the display object 1858. The video control unit 1160 simply performs control to display the video of the generated display object 1857 and the display object 1858 in the display area 1501 of FIG. 18E.

[0344] The display object 1851 of the character, which is the main content, may use a character image rendered from a 3D model. Alternatively, a 2D animated character may be used. Alternatively, a live-action image of a person or the like may be used as the character image. A music promotional video of a character or person singing may be used as the display object of the character of the display object 1851.

[0345] 18D, display objects 1857 and 1858, which are secondary content, are examples of display objects of letters that are the lyrics of the song being sung by the character, but are not limited to this and may be display objects of so-called effect images that are displayed in front of the character. Specific examples of effect images may be any display object that displays an effect in front of the character, such as an effect image that displays stars that display brilliance, an effect image that displays lightning, an effect image that displays rain, an effect image that displays falling snow, or an effect image that displays fluttering flower petals. These effect images may also be displayed in conjunction with the animation of the character, which is the primary content.

[0346] Even when an effect image is displayed on display object 1857 or display object 1858, which is secondary content, display object 1857 or display object 1858 is displayed in a space-floating image with a different depth from character display object 1851, which is the primary content, and motion parallax occurs. This has the effect of making it less susceptible to the constraint that display objects such as characters are perceived as two-dimensional, even if the display object of the effect image is superimposed in front of the character display object.

[0347] 18D and 18E, in a space-floating image display device that displays multiple layers of space-floating images with different protrusion amounts, it is possible to display text information and effect images, which are secondary content, in a display position with a different depth direction relative to a display object such as a character, which is the main content. This makes it possible to reduce the constraint of recognizing a display object such as a character as a two-dimensional object, even if the display object such as a text display object or an effect image display object is displayed in a position where it is superimposed on the display object such as a character, which is more preferable.

[0348] 18D and 18E, an example has been described in which space floating image 3-1 and space floating image 3-2, which have different depths, are used to display character display object 1851 as space floating image 3-2, and character display object 1857 and character display object 1858 as space floating image 3-1. This allows the user to easily recognize that character display object 1857 and character display object 1858 are displayed in front of character display object 1851.

[0349] On the other hand, as a modified example, an operation menu object may be displayed as the space-floating image 3-1 instead of or in addition to the character display object. The operation menu object may be an operation menu for operating the space-floating image display device 1000. An example of the operation of the space-floating image display device 1000 is an operation for adjusting the output audio level of the audio output unit 1140, a mute operation, etc.

[0350] Furthermore, the object of the operation menu may be an operation menu for performing an operation related to the display of the character display object 1851, which is the main content. Examples of the operation related to the display of the display object 1851 include an operation to switch the character, an operation to change the shape or color of the character's costume, or an operation to change the character's movement. Other examples of the operation related to the display of the display object 1851 may be an operation to change the display brightness of the display object 1851, an operation to change the display position, or an operation to change the display size, etc.

[0351] For example, if the aerial operation detection sensor 1351E shown in Fig. 17C is configured to detect user operations on the space-floating image 3-1 in front of multiple space-floating images with different depths, it becomes possible to operate objects in the operation menu to perform various operations. With this configuration, it becomes possible to use user operations via the operation menu displayed on one space-floating image of multiple space-floating images with different depths to operate the display of another space-floating image with a different depth from the one space-floating image.

[0352] Furthermore, the motion parallax effect caused by viewpoint movement allows the user to more clearly recognize that the operation menu object displayed on the floating image 3-1 in the foreground and the main content object displayed on the floating image 3-2 in the background are displayed at multiple display positions with different depth directions.

[0353] The technology according to this embodiment displays high-resolution, high-brightness image information in a state where it floats in space, allowing users to operate the system without feeling anxious about contact infection. If the technology according to this embodiment is used in a system used by an unspecified number of users, it will be possible to reduce the risk of contact infection and provide a contactless user interface that can be used without anxiety. This will contribute to achieving the three Sustainable Development Goals (SDGs) advocated by the United Nations: "Ensure good health and well-being for all." Contribute to "welfare."

[0354] Furthermore, the technology according to this embodiment reduces the divergence angle of the emitted image light and aligns it with a specific polarization, thereby efficiently reflecting only the normal reflected light from the retroreflector, thereby enabling a bright and clear floating image in space with high light utilization efficiency. The technology according to this embodiment can provide a highly usable non-contact user interface that can significantly reduce power consumption. This will contribute to achieving the "9 Sustainable Development Goals (SDGs)" advocated by the United Nations, "Build resilient infrastructure, promote inclusive and sustainable industrialization, and foster innovation." and contribute to "11. Creating sustainable communities."

[0355] Various embodiments have been described above in detail. However, the present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments are detailed descriptions of the entire system to clearly explain the present invention, and the present invention is not necessarily limited to a system including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations. [Explanation of symbols]

[0356] 1...display device, 2...retroreflector (retroreflector), 3...spatial image (space-floating image), 105...window glass, 100...transparent member, 101...polarized light separation member, 101B...polarized light separation member, 12...absorptive polarizer, 13...light source device, 54...light direction conversion panel, 151...retroreflector, 102, 202...LED substrate, 203...light guide, 205, 271...reflective sheet, 206, 270...phase difference plate, 230...user, 1000...space-floating image display device, 1110...control unit, 1160...image control unit, 1180...imaging unit, 1102...image display unit, 1350...air operation detection unit, 1351...air operation detection sensor.

Claims

1. A floating-in-the-air image display device that displays floating-in-the-air images, a display unit that displays an image; a first polarization separation member; a first λ / 4 plate; A first retroreflector; a second polarization separating member; a second λ / 4 plate; and A specular reflector; a third λ / 4 plate; and A second retroreflector; Equipped with The display screen of the display unit has a first image display area and a second image display area, The display unit, the first polarization separation member, the first λ / 4 plate, the first retroreflector, the second polarization separation member, the second λ / 4 plate, the specular reflector, the third λ / 4 plate, and the second retroreflector, Image light of a predetermined polarization emitted from a first image display area of ​​the display screen of the display unit passes through the first polarization separation member, the transmitted image light passes through the first λ / 4 plate and is retroreflected by the first retroreflector, the image light retroreflected by the first retroreflector passes through the first λ / 4 plate to become image light of the other polarization whose phase is different by 90° from the predetermined polarization, and the image light of the other polarization is reflected by the first polarization separation member to form a first floating image in the air, which is a real image; Image light of a predetermined polarization emitted from a second image display area of ​​the display screen of the display unit passes through the second polarization separation member, the transmitted image light passes through the second λ / 4 plate and is reflected by the specular reflector, the image light reflected by the specular reflector passes through the second λ / 4 plate to become image light of another polarization whose phase is different by 90° from the predetermined polarization, the image light of the other polarization is reflected by the second polarization separation member, passes through the third λ / 4 plate and is retroreflected by the second retroreflector, and is then reflected by the second retroreflector. The image light retroreflected by the reflecting plate passes through the third λ / 4 plate, becoming the image light of the predetermined polarization whose phase differs by 90° from the other polarization, the image light of the predetermined polarization traveling from the third λ / 4 plate to the second polarization separating member passes through the second polarization separating member, the image light of the predetermined polarization traveling from the second polarization separating member to the first polarization separating member passes through the first polarization separating member, and the image light of the predetermined polarization traveling through the first polarization separating member forms a second floating image in the air, which is a real image. It is arranged as follows: A floating video display device.

2. 2. The floating-in-the-air image display device according to claim 1, the first floating-in-the-air image and the second floating-in-the-air image form a floating-in-the-air image with multiple layers having different depths as seen from the user; A floating video display device.

3. 2. The floating-in-the-air image display device according to claim 1, The first floating image, the second floating image, the first polarization separating member, the second polarization separating member, and the second retroreflector are arranged on the same straight line. A floating video display device.

4. 4. The airborne image display device according to claim 3, The display unit, the first retroreflector, and the specular reflector are arranged at positions that are not aligned with each other. A floating video display device.

5. A floating-in-the-air image display device that displays floating-in-the-air images, a display unit that displays an image; a λ / 2 plate; a first polarization separation member; a first λ / 4 plate; A first retroreflector; a second polarization separating member; a second λ / 4 plate; and A second retroreflector; Equipped with The display screen of the display unit has a first image display area and a second image display area, The display unit, the λ / 2 plate, the first polarization separation member, the first λ / 4 plate, the first retroreflector, the second polarization separation member, the second λ / 4 plate, and the second retroreflector, Image light of a predetermined polarization emitted from a first image display area of ​​the display screen of the display unit passes through the first polarization separation member, the transmitted image light passes through the first λ / 4 plate and is retroreflected by the first retroreflector, the image light retroreflected by the first retroreflector passes through the first λ / 4 plate to become image light of the other polarization whose phase is different by 90° from the predetermined polarization, and the image light of the other polarization is reflected by the first polarization separation member to form a first floating image in the air, which is a real image; Image light of a predetermined polarization emitted from a second image display area of ​​the display screen of the display unit passes through the λ / 2 plate to become image light of the other polarization whose phase is different by 90° from the predetermined polarization, is reflected by the second polarization separation member, the reflected image light passes through the second λ / 4 plate to be retroreflected by the second retroreflector, the image light retroreflected by the second retroreflector passes through the second λ / 4 plate to become image light of the predetermined polarization whose phase is different by 90° from the other polarization, the image light of the predetermined polarization traveling from the second λ / 4 plate to the second polarization separation member passes through the second polarization separation member, the image light of the predetermined polarization traveling from the second polarization separation member to the first polarization separation member passes through the first polarization separation member, and the image light of the predetermined polarization that has passed through the first polarization separation member forms a second floating image in the air, which is a real image. It is arranged as follows: A floating video display device.

6. 6. The airborne image display device according to claim 5, the first floating-in-the-air image and the second floating-in-the-air image form a floating-in-the-air image with multiple layers having different depths as seen from the user; A floating video display device.

7. 6. The airborne image display device according to claim 5, The first floating image, the second floating image, the first polarization separating member, the second polarization separating member, and the second retroreflector are arranged on the same straight line. A floating video display device.

8. 8. The airborne image display device according to claim 7, The display unit and the first retroreflector are disposed at positions that are not aligned with each other. A floating video display device.

9. 6. The airborne image display device according to claim 5, an angle control sheet that shifts the traveling direction of light by a predetermined angle is attached to a display surface of the display unit, and the display unit is disposed at an angle corresponding to the predetermined angle at which the angle control sheet shifts the traveling direction of light; Since the display unit is disposed at an angle, the optical path length of the image light emitted from the first image display area of ​​the display screen of the display unit to the first retroreflector is shorter than the optical path length of the image light emitted from the second image display area of ​​the display screen of the display unit to the second retroreflector. A floating video display device.

10. A floating-in-the-air image display device that displays floating-in-the-air images, a display unit that displays an image; a first polarization separation member; a λ / 4 plate; A retroreflective plate; a second polarization separating member; a λ / 2 plate; Equipped with The display screen of the display unit has a first image display area and a second image display area, The display unit, the first polarization separation member, the λ / 4 plate, the retroreflector, the second polarization separation member, and the λ / 2 plate, Image light of a predetermined polarization emitted from a first image display area of ​​the display screen of the display unit passes through the first polarization separation member, the transmitted image light passes through the λ / 4 plate and is retroreflected by the retroreflector, the image light retroreflected by the retroreflector passes through the λ / 4 plate to become image light of the other polarization whose phase is different by 90° from the predetermined polarization, and the image light of the other polarization is reflected by the first polarization separation member to form a first floating image in the air, which is a real image; Image light of a predetermined polarization emitted from a second image display area of ​​the display screen of the display unit passes through the second polarization separation member, the transmitted image light passes through the λ / 4 plate and is retroreflected by the retroreflector, the image light retroreflected by the retroreflector passes through the λ / 4 plate to become image light of the other polarization whose phase is different by 90° from the predetermined polarization, the image light of the other polarization is reflected by the second polarization separation member, the image light reflected by the second polarization separation member passes through the λ / 2 plate to become image light of the predetermined polarization whose phase is different by 90° from the other polarization, the image light of the predetermined polarization traveling from the λ / 2 plate to the first polarization separation member passes through the first polarization separation member, and the image light of the predetermined polarization that passed through the first polarization separation member forms a second floating image in the air, which is a real image. It is arranged as follows: A floating video display device.

11. The airborne image display device according to claim 10, the first floating-in-the-air image and the second floating-in-the-air image form a floating-in-the-air image with multiple layers having different depths as seen from the user; A floating video display device.

12. The airborne image display device according to claim 10, the first floating-in-the-air image, the second floating-in-the-air image, the first polarization separation member, the λ / 2 plate, and the second polarization separation member are arranged on the same straight line; A floating video display device.

13. 13. The airborne image display device according to claim 12, The display unit and the retroreflector are disposed at positions that are not aligned with each other. A floating video display device.

14. The airborne image display device according to claim 10, an angle control sheet that shifts the traveling direction of light by a predetermined angle is attached to a display surface of the display unit, and the display unit is disposed at an angle corresponding to the predetermined angle at which the angle control sheet shifts the traveling direction of light; Since the display unit is disposed at an angle, the optical path length of the image light emitted from the first image display area of ​​the display screen of the display unit to the retroreflector is shorter than the optical path length of the image light emitted from the second image display area of ​​the display screen of the display unit to the retroreflector. A floating video display device.

15. A floating-in-the-air image display device that displays floating-in-the-air images, a display unit that displays an image; a first polarization separation member; a second polarization separating member; one or more retroreflectors; Equipped with The display screen of the display unit has a first image display area and a second image display area, Image light emitted from a first image display area of ​​the display screen of the display unit is transmitted through the first polarization separation member and retroreflected by any one of the one or more retroreflectors, and then forms a first floating-in-the-air image in the air; and image light emitted from a second image display area of ​​the display screen of the display unit is transmitted through or reflected by the second polarization separation member and retroreflected by any one of the one or more retroreflectors, and then forms a second floating-in-the-air image in the air, whereby the first floating-in-the-air image and the second floating-in-the-air image form a multi-layer floating-in-the-air image with different depth directions as viewed from the user. A floating video display device.

16. 16. The airborne image display device according to claim 15, a video processing unit that processes video images to be displayed on the display unit; the image processing unit, when displaying a first display object in the first floating-in-the-air image and a second display object in the second floating-in-the-air image, performs image processing to display a virtual shadow of a display object displayed in one of the first floating-in-the-air image and the second floating-in-the-air image, which is positioned vertically above a space in which the floating-in-the-air image display device is installed, in a partial area of ​​a display object displayed in the other floating-in-the-air image which is positioned vertically below the space in which the floating-in-the-air image display device is installed; A floating video display device.

17. 17. The airborne image display device according to claim 16, The image processing for displaying the virtual shadow performed by the image processing unit is image processing for displaying a partial area of ​​the display object displayed in the other floating-in-the-air image in black, image processing for reducing the luminance of the image signal, or image processing for reducing the saturation of the image signal. A floating video display device.

18. 16. The airborne image display device according to claim 15, In one of the floating-in-the-air images of the plurality of layers that is displayed on the front side as seen from the user, a first display object that is a primary content is displayed near the center in the horizontal direction as seen from the user, and in one of the floating-in-the-air images of the plurality of layers that is displayed on the back side as seen from the user, a second display object that is a secondary content is displayed at a position that avoids the center in the horizontal direction as seen from the user. A floating video display device.

19. 19. The airborne image display device according to claim 18, the first display object that is the main content is a character object; A floating video display device.

20. 20. The airborne image display device according to claim 19, the second display object that is the secondary content is a background object located behind a character; A floating video display device.

21. 16. The airborne image display device according to claim 15, a first display object, which is a primary content, is displayed in one of the plurality of layers of floating-in-the-air images that is displayed on the far side as viewed from the user, and a second display object, which is a secondary content and is an object disposed in front of the first display object, is displayed in one of the plurality of layers of floating-in-the-air images that is displayed on the near side as viewed from the user; A floating video display device.

22. 22. The airborne image display device according to claim 21, the first display object that is the main content is a character object; A floating video display device.

23. 23. The airborne image display device according to claim 22, the second display object that is the subsidiary content is a character object; A floating video display device.

24. 23. The airborne image display device according to claim 22, the second display object, which is the subsidiary content, is an effect image object; A floating video display device.

25. 16. The airborne image display device according to claim 15, a light blocking plate that separates an optical path of image light that is emitted from a first image display area of ​​a display screen of the display unit and proceeds toward the first polarization separating member from an optical path of image light that is emitted from a second image display area of ​​the display screen of the display unit and proceeds toward the second polarization separating member, a predetermined gap area is provided between the first image display area and the second image display area on the display screen of the display unit, the predetermined gap area is a content non-display area, and the width of the predetermined gap is longer than the thickness of the light blocking plate; A floating video display device.

Images