Floating image display device

Abstract

To provide a more appropriate floating image display device which contributes to sustainable development goals (SDGs) of "3. good health and well-being for all," "9. building a foundation for industry and technological innovation," and "11. creating sustainable cities."SOLUTION: A floating image display device is provided, comprising at least a display unit movement mechanism for moving a display unit, and a sensor movement mechanism for moving a sensor. Moving the display unit using the display unit movement mechanism causes an imaging position of a floating image to move in an output direction of image light forming the floating image, and the sensor is moved using the sensor movement mechanism so as to match the imaging position of the floating image.SELECTED DRAWING: Figure 20A

Description

[Technical Field]

[0001] The present invention relates to a floating-in-the-air image display device. [Background technology]

[0002] The floating information display technology is disclosed in, for example, Patent Document 1. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-128722 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the disclosure of Patent Document 1 does not sufficiently consider a configuration for obtaining practical brightness and quality for the floating image, or a configuration for allowing the user to view the floating image more enjoyably.

[0005] An object of the present invention is to provide a more suitable floating-in-the-air image display device. [Means for solving the problem]

[0006] To solve the above problems, for example, the configuration described in the claims is adopted. The present application includes multiple means for solving the above problems, and one example thereof may be configured as follows: A floating-in-the-air image display device, comprising: an image processing unit that processes image; a display unit that displays image processed by the image processing unit; an optical system that generates a floating-in-the-air image based on the image displayed by the display unit; a sensor that detects a user's mid-air operation on the floating-in-the-air image; a display unit movement mechanism that moves the display unit along the emission direction of the image; and a sensor movement mechanism that moves the sensor, wherein by moving the display unit with the display unit movement mechanism, the position of the image of the floating-in-the-air image is moved along the emission direction of the image light that forms the floating-in-the-air image, and the sensor movement mechanism moves the sensor to align it with the position of the image of the floating-in-the-air image. [Effects of the Invention]

[0007] According to the present invention, a more suitable floating-in-the-air image display device can be realized. Other problems, configurations, and effects will become clear in the following description of the embodiments. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram showing an example of a usage form of a space floating image display device according to an embodiment of the present invention; [Figure 2A] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 2B] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 2C] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 2D] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a floating-in-the-air image display device according to an embodiment of the present invention; [Figure 2E] 1 is a projection view of a retroreflector constituting a floating-in-the-air image display device according to an embodiment of the present invention; [Figure 2F] 1 is a top view of a retroreflector constituting a floating-in-the-air image display device according to an embodiment of the present invention; FIG. [Figure 2G] FIG. 1 is a perspective view showing a corner reflector that constitutes a retroreflector that constitutes a floating-in-the-air image display device according to an embodiment of the present invention. [Figure 2H] FIG. 1 is a top view showing a corner reflector that constitutes a retroreflector that constitutes a floating-in-the-air image display device according to an embodiment of the present invention. [Figure 2I] FIG. 1 is a side view showing a corner reflector that constitutes a retroreflector that constitutes a floating-in-the-air image display device according to an embodiment of the present invention. [Figure 3] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4A] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4B] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4C] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4D] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4E] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4F] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4G] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4H] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4I] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4J] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4K]1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4L] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4M] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4N] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4O] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 4P] 1 is a diagram showing an example of the configuration of a space floating image display device according to an embodiment of the present invention; [Figure 5] 1 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. [Figure 6] 1 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. [Figure 7] 1 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. [Figure 8] 1 is a layout diagram showing a main part of a space floating image display device according to an embodiment of the present invention; [Figure 9] 1 is a cross-sectional view showing a configuration of a display device according to an embodiment of the present invention. [Figure 10] 1 is a cross-sectional view showing a configuration of a display device according to an embodiment of the present invention. [Figure 11] 1 is an explanatory diagram for explaining the light source diffusion characteristics of an image display device according to an embodiment of the present invention. [Figure 12] 1 is an explanatory diagram for explaining the diffusion characteristics of a video display device according to an embodiment of the present invention; [Figure 13A] 1 is a diagram illustrating an example of a problem to be solved by image processing according to an embodiment of the present invention; [Figure 13B] FIG. 10 is an explanatory diagram of an example of image processing according to an embodiment of the present invention. [Figure 13C] FIG. 10 is an explanatory diagram of an example of a video display process according to an embodiment of the present invention. [Figure 13D] FIG. 10 is an explanatory diagram of an example of a video display process according to an embodiment of the present invention. [Figure 14] 1 is a diagram showing an example of a main part configuration and a retroreflection part configuration of a space floating image display device according to an embodiment of the present invention; [Figure 15] FIG. 10 is a diagram showing a configuration example of a space floating image display device inside a vehicle in a comparative example. [Figure 16] 1 is a diagram showing an example of the configuration of a space floating image display device installed in a vehicle according to an embodiment; [Figure 17] 1 is a diagram illustrating a configuration example of a space floating image display device according to an embodiment; [Figure 18] FIG. 1 is a diagram showing a basic operation flow of a space floating image display device according to an embodiment. [Figure 19A] 10A and 10B are diagrams illustrating an example of a device angle adjustment mechanism of the space floating image display device according to one embodiment. [Figure 19B] 10A and 10B are diagrams illustrating an example of a device angle adjustment mechanism of the space floating image display device according to one embodiment. [Figure 20A] 1 is a diagram illustrating a configuration example of a space floating image display device according to an embodiment; [Figure 20B] 1 is a diagram illustrating a configuration example of a space floating image display device according to an embodiment; [Figure 20C] 1 is a diagram illustrating a configuration example of a space floating image display device according to an embodiment; [Figure 21A] 10A and 10B are diagrams illustrating examples of changing the device angle of the space floating image display device according to an embodiment. [Figure 21B] 10A and 10B are diagrams illustrating examples of changing the device angle of the space floating image display device according to an embodiment. [Figure 22] 10A and 10B are diagrams illustrating examples of differences in image formation positions in a space floating image display device according to an embodiment. [Figure 23] 1 is a diagram showing a configuration example of an actuator for a display unit in a space floating image display device according to an embodiment; [Figure 24] 1 is a diagram showing a configuration example of a sensor actuator in a space floating image display device according to an embodiment; [Figure 25]FIG. 10 is a diagram showing an example of the configuration of an aerial operation detection sensor in the space floating image display device according to one embodiment. [Figure 26] FIG. 2 is a diagram illustrating a configuration example for control by an image processing unit in a space floating image display device according to an embodiment. [Figure 27A] 1 is a diagram showing an example of a display screen of a space floating image in a space floating image display device according to an embodiment; [Figure 27B] 1 is a diagram showing an example of a display screen of a space floating image in a space floating image display device according to an embodiment; [Figure 28] FIG. 10 is a diagram showing state transitions of a control example in a space floating image display device according to an embodiment. [Figure 29] FIG. 10 is a diagram showing state transitions of a control example in a space floating image display device according to an embodiment. [Figure 30] FIG. 10 is a diagram showing an example of control in a space floating image display device according to a modified example. [Figure 31] FIG. 10 is a diagram showing an example of control in a space floating image display device according to a modified example. [Figure 32] FIG. 10 is a diagram showing a control flow in a space floating image display device according to a modified example. [Figure 33] FIG. 10 is a diagram showing an example of control in a space floating image display device according to a modified example. [Figure 34] 1 is an explanatory diagram relating to a cut-off of a screen of a space floating image in a space floating image display device according to an embodiment; [Figure 35A] 10A and 10B are diagrams illustrating an example of a screen cutoff of a space-floating image in a space-floating image display device according to an embodiment. [Figure 35B] 10A and 10B are diagrams illustrating an example of a screen of a space floating image related to control of display magnification in a space floating image display device according to an embodiment. [Figure 36] 10A and 10B are diagrams illustrating an example of a screen of a liquid crystal display panel regarding control of display magnification in a space floating image display device according to an embodiment. [Figure 37] 10A and 10B are diagrams illustrating examples of buttons on an operation input unit of a housing in the space floating image display device according to one embodiment. [Figure 38] FIG. 10 is a diagram showing an example of control in a space floating image display device according to a modified example. [Figure 39] FIG. 10 is a diagram showing an example of control in a space floating image display device according to a modified example. [Figure 40] 1 is a diagram illustrating a configuration example of a space floating image display device according to an embodiment; [Figure 41] 10A and 10B are diagrams illustrating examples of buttons on an operation input unit of a housing in the space floating image display device according to one embodiment. [Figure 42] 10A and 10B are diagrams illustrating an example of control in a space floating image display device according to an embodiment. [Figure 43] FIG. 10 is a diagram showing state transitions of a control example in a space floating image display device according to an embodiment. [Figure 44] FIG. 10 is a diagram illustrating another example of control in the space floating image display device according to an embodiment. [Figure 45] 1 is a diagram showing an example of a screen of a space floating image in a space floating image display device according to an embodiment; [Figure 46] 10A and 10B are diagrams illustrating an example of control in a space floating image display device according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the description of the embodiments, and various changes and modifications can be made by those skilled in the art within the scope of the technical ideas disclosed in this specification. Furthermore, in all drawings used to explain the present invention, components having the same functions are given the same reference numerals, and repeated explanations thereof may be omitted.

[0010] The following examples relate to an image display device that can transmit an image generated by image light from an image light source through a transparent member that separates a space, such as glass, and display the image as a floating image outside the transparent member. In the following explanation of the examples, the image floating in space is expressed using the term "floating image in space." Instead of this term, it is also acceptable to express it as "aerial image," "spatial image," "floating image in space," "floating optical image of displayed image," "floating optical image of displayed image," etc. The term "floating image in space," which is mainly used in the explanation of the examples, is used as a representative example of these terms.

[0011] According to the following embodiments, an image display device suitable for, for example, bank ATMs, train station ticket machines, digital signage, and the like can be realized. For example, currently, bank ATMs, train station ticket machines, and the like typically use touch panels. However, by using a transparent glass surface or a light-transmitting plate, high-resolution image information can be displayed in a floating state on the glass surface or light-transmitting plate. In this case, by making the divergence angle of the emitted image light small, i.e., an acute angle, and further aligning it with a specific polarization, only the normal reflected light is efficiently reflected by the retroreflector. This improves light utilization efficiency and suppresses the ghost images that occur in addition to the main floating image, which is a problem with conventional retroreflection systems, thereby achieving a clear floating image. Furthermore, by using a device including the light source of this embodiment, a novel and highly usable floating image display device (floating image display system) can be provided that can significantly reduce power consumption. Furthermore, a floating image display device for a vehicle can be provided that can display a so-called unidirectional floating image that can be viewed inside and / or outside the vehicle.

[0012] Example 1 An example of the configuration of a space floating image display device will be described below as a first embodiment of the present invention.

[0013] <Example of how to use the space floating image display device> FIG. 1 is a diagram showing an example of a usage form of a space-floating image display device according to an embodiment of the present invention, and is a diagram showing the overall configuration of the space-floating image display device according to this embodiment. The specific configuration of the space-floating image display device will be described in detail using FIG. 2 and other figures. Light with a narrow-angle directional characteristic and specific polarization is emitted from image display device 1 as an image light beam, reflected by the optical system within the space-floating image display device, and then incident on retroreflector 2. It is then retroreflected and transmitted through transparent member 100 (glass, etc.), forming a real aerial image (space-floating image 3) on the outside of the glass surface. In the following embodiments, the retroreflector 2 (retroreflector) is used as an example of a retroreflector. However, the retroreflector 2 of the present invention is not limited to a planar plate, and is used as an example of a concept including a sheet-like retroreflector attached to a planar or non-planar member, or an entire assembly in which a sheet-like retroreflector is attached to a planar or non-planar member. Furthermore, since the light rays reflected by the retroreflector 2 have the optical property of forming an image, the retroreflector 2 may also be expressed as an imaging optical member or an imaging optical plate.

[0014] In addition, in a store or the like, a space is partitioned by a show window (also called "window glass") 105, which is a translucent member such as glass. According to the space floating image display device of this embodiment, it is possible to transmit such a transparent member and display a floating image in one direction to the outside and / or inside of the store (space).

[0015] 1, the inside of the window glass 105 (inside the store) is shown in the depth direction, and the outside (for example, the sidewalk) is shown in the foreground. On the other hand, by providing a means for reflecting specific polarized waves on the window glass 105, it is possible to reflect the waves and form an aerial image at a desired position inside the store.

[0016] <Configuration example of optical system for space floating image display device> 2A is a diagram showing an example of the configuration of an optical system of a space-floating image display device according to one embodiment of the present invention. The configuration of the space-floating image display device will be described in more detail using FIG. 2A. As shown in FIG. 2A(1), a display device 1 that diverges specific polarized image light at a narrow angle is provided in an oblique direction of a transparent member 100 such as glass. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates specific polarized light with narrow-angle diffusion characteristics.

[0017] Image light of a specific polarization from the display device 1 is reflected by a polarization separator 101 (in the figure, the polarization separator 101 is formed into a sheet and adhered to the transparent member 100) that has a film that selectively reflects image light of a specific polarization and is provided on a transparent member 100, and then enters the retroreflector 2. A λ / 4 plate 21 is provided on the image light incident surface of the retroreflector 2. The image light passes through the λ / 4 plate 21 twice, once when it enters the retroreflector 2 and once when it exits, thereby undergoing polarization conversion from the specific polarization to the other polarization. Here, the polarization separator 101 that selectively reflects image light of a specific polarization has the property of transmitting the polarized light of the other polarization that has been polarization-converted, so the image light of the specific polarization after polarization conversion passes through the polarization separator 101. The image light that has passed through the polarization separator 101 forms a space-floating image 3, which is a real image, outside the transparent member 100. 2A shows an example in which the chief ray of the image light incident on the retroreflector 2 is incident at an angle of 90° to the retroreflector 2. However, the incident angle of the chief ray of the image light on the retroreflector 2 is not limited to 90°, and an angle of, for example, 90°±15° can also be used.

[0018] Here, a first example of polarization design for the optical system of FIG. 2A will be described. For example, S-polarized (S stands for Senkrecht; polarized light whose electric field oscillates perpendicular to the plane of incidence) image light may be emitted from display device 1 to polarization separator 101, which may have the property of reflecting S-polarized light and transmitting P-polarized (P stands for parallel; polarized light whose electric field oscillates within the plane of incidence) light. In this case, the S-polarized image light reaching polarization separator 101 from display device 1 is reflected by polarization separator 101 and travels toward retroreflector 2. When the image light is reflected by retroreflector 2, it passes twice through λ / 4 plate 21 provided on the incident surface of retroreflector 2, and is therefore converted from S-polarized to P-polarized light. The P-polarized image light then travels toward polarization separator 101 again. Here, the polarization separation member 101 has the property of reflecting S-polarized light and transmitting P-polarized light, so the P-polarized image light passes through the polarization separation member 101 and then through the transparent member 100. The image light that passes through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a space-floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101. This polarization design allows the space-floating image 3 to be formed optimally.

[0019] Next, a second example of polarization design for the optical system of FIG. 2A will be described. For example, P-polarized image light may be emitted from display device 1 to polarization separator 101, which may have the property of reflecting P-polarized light and transmitting S-polarized light. In this case, the P-polarized image light reaching polarization separator 101 from display device 1 is reflected by polarization separator 101 and travels toward retroreflector 2. When the image light is reflected by retroreflector 2, it passes twice through λ / 4 plate 21 provided on the incident surface of retroreflector 2, converting the P-polarized light to S-polarized light. The S-polarized image light then travels back toward polarization separator 101. Here, polarization separator 101 has the property of reflecting P-polarized light and transmitting S-polarized light, so the S-polarized image light passes through polarization separator 101 and then through transparent member 100. The image light transmitted through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101. This polarization design allows the floating image 3 to be formed optimally.

[0020] The light that forms the floating image 3 is a collection of light rays that converge from the retroreflector 2 to the optical image of the floating image 3, and these light rays continue to travel in a straight line even after passing through the optical image of the floating image 3. Therefore, the floating image 3 is a highly directional image, unlike the diffused image light formed on a screen by a general projector or the like. Therefore, in the configuration of FIG. 2A, when a user views the floating image 3 from the direction of arrow A, the floating image 3 is perceived as a bright image. However, when another person views the floating image 3 from the direction of arrow B, the floating image 3 cannot be perceived as an image at all. This characteristic is very suitable for use in a system that displays images that require high security or highly confidential images that should be kept secret from people directly facing the user.

[0021] Depending on the performance of the retroreflector 2, the polarization axis of the reflected image light may become irregular. The reflection angle may also become irregular. Such irregular light may not maintain the polarization state and propagation angle assumed in the design. For example, light with an unintended polarization state and propagation angle may re-enter the image display surface of the liquid crystal display panel 11 directly from the position of the retroreflector 2 without passing through a polarization separation member. Such light with an unintended polarization state and propagation angle may re-enter the image display surface of the liquid crystal display panel 11 after being reflected by components within the space-floating image display device. Such light re-entering the image display surface of the liquid crystal display panel 11 may be re-reflected by the image display surface of the liquid crystal display panel 11 constituting the display device 1, potentially generating ghost images and degrading the image quality of the space-floating image. Therefore, in this embodiment, an absorbing polarizer 12 may be provided on the image display surface of the display device 1. The image light emitted from the display device 1 is transmitted through the absorptive polarizer 12, and the reflected light returning from the polarization separation member 101 is absorbed by the absorptive polarizer 12, thereby suppressing the re-reflection. This makes it possible to prevent degradation of image quality due to ghost images of spatially floating images. Specifically, if the display device 1 is configured to emit S-polarized image light to the polarization separation member 101, the absorptive polarizer 12 may be a polarizer that absorbs P-polarized light. Furthermore, if the display device 1 is configured to emit P-polarized image light to the polarization separation member 101, the absorptive polarizer 12 may be a polarizer that absorbs S-polarized light.

[0022] The polarization separation member 101 may be formed of, for example, a reflective polarizing plate or a metal multilayer film that reflects specific polarized waves.

[0023] Next, Fig. 2A(2) shows an example of the surface shape of a typical retroreflector 2. A prism body with a regularly arranged array of triangular pyramidal recessed reflective surfaces is arranged on the retroreflector 2. Light rays incident on the arranged triangular pyramidal recesses are reflected by the multiple reflective surfaces of the triangular pyramidal recesses and emitted as retroreflected light in a direction corresponding to the incident light, and a real floating image is displayed based on the image displayed on the display device 1.

[0024] The surface shape of the retroreflector according to this embodiment is not limited to the above example. Various surface shapes that achieve retroreflection may be used. Specifically, the surface of the retroreflector according to this embodiment may be provided with retroreflection elements in which triangular pyramidal prisms, hexagonal pyramidal prisms, other polygonal prisms, multi-vertex prisms, or combinations thereof are periodically arranged. Alternatively, the surface of the retroreflector according to this embodiment may be provided with retroreflection elements in which these prisms are periodically arranged to form cube corners. These may also be referred to as corner reflector arrays or polyhedral reflector arrays. Alternatively, the surface of the retroreflector according to this embodiment may be provided with capsule lens-type retroreflection elements in which glass beads are periodically arranged. The detailed configuration of these retroreflection elements can be achieved using existing technology, so a detailed description will be omitted. Specifically, the techniques disclosed in Japanese Patent Laid-Open Nos. 2001-33609, 2001-264525, 2005-181555, 2008-70898, and 2009-229942 may be used.

[0025] <Another configuration example 1 of the optical system of the space floating image display device> Another example of the configuration of the optical system of the space floating image display device will be explained using Fig. 2B. In Fig. 2B, components with the same reference numerals as Fig. 2A have the same functions and configurations as Fig. 2A. For the sake of simplicity, repeated explanations of such components will be omitted.

[0026] In the optical system of FIG. 2B, as in FIG. 2A, image light of a specific polarization is output from the display device 1. The image light of a specific polarization output from the display device 1 is input to a polarization separator 101B. The polarization separator 101B is a member that selectively transmits image light of a specific polarization. Unlike the polarization separator 101 of FIG. 2A, the polarization separator 101B is not integrated with the transparent member 100 but has an independent plate-like shape. Therefore, the polarization separator 101B may also be referred to as a polarization separator plate. The polarization separator 101B may be configured as a reflective polarizer configured by attaching a polarization separator sheet to a transparent member. Alternatively, the transparent member may be formed of a metal multilayer film that selectively transmits specific polarization and reflects polarization of other specific polarizations. In FIG. 2B, the polarization separator 101B is configured to transmit image light of a specific polarization output from the display device 1.

[0027] The image light that has passed through the polarization separation member 101B is incident on the retroreflector 2. A λ / 4 plate 21 is provided on the image light incident surface of the retroreflector. The image light is polarized and converted from a specific polarization to the other polarization by passing through the λ / 4 plate 21 twice, once when it enters the retroreflector and once when it leaves. Here, the polarization separation member 101B has the property of reflecting the polarized light of the other polarization that has been polarized and converted by the λ / 4 plate 21, so the image light after polarization conversion is reflected by the polarization separation member 101B. The image light reflected by the polarization separation member 101B passes through the transparent member 100 and forms a space-floating image 3, which is a real image, outside the transparent member 100.

[0028] Here, a first example of polarization design for the optical system of FIG. 2B will be described. For example, a configuration may be adopted in which P-polarized image light is emitted from display device 1 to polarization separator 101B, and polarization separator 101B has the property of reflecting S-polarized light and transmitting P-polarized light. In this case, the P-polarized image light that reaches polarization separator 101B from display device 1 passes through polarization separator 101B and proceeds to retroreflector 2. When the image light is reflected by retroreflector 2, it passes twice through λ / 4 plate 21 provided on the incident surface of retroreflector 2, and the image light is converted from P-polarized light to S-polarized light. The image light converted to S-polarized light proceeds again to polarization separator 101B. Here, polarization separator 101B has the property of reflecting S-polarized light and transmitting P-polarized light, so the S-polarized image light is reflected by polarization separator 101 and passes through transparent member 100. The image light transmitted through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a space-floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101B. This polarization design allows the space-floating image 3 to be formed optimally.

[0029] Next, a second example of polarization design for the optical system of FIG. 2B will be described. For example, S-polarized image light may be emitted from display device 1 to polarization separator 101B, which may have the property of reflecting P-polarized light and transmitting S-polarized light. In this case, the S-polarized image light reaching polarization separator 101B from display device 1 passes through polarization separator 101B and proceeds to retroreflector 2. When the image light is reflected by retroreflector 2, it passes twice through λ / 4 plate 21 provided on the incident surface of retroreflector 2, converting the image light from S-polarized light to P-polarized light. The P-polarized image light then proceeds again to polarization separator 101B. Here, polarization separator 101B has the property of reflecting P-polarized light and transmitting S-polarized light, so the P-polarized image light is reflected by polarization separator 101 and passes through transparent member 100. The image light transmitted through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a space-floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101B. This polarization design allows the space-floating image 3 to be formed optimally.

[0030] In FIG. 2B , the image display surface of the display device 1 and the surface of the retroreflector 2 are arranged parallel to each other. The polarization separator 101B is arranged tilted at an angle α (e.g., 30°) relative to the image display surface of the display device 1 and the surface of the retroreflector 2. When the polarization separator 101B reflects the image light, the direction of the image light reflected by the polarization separator 101B (the direction of the chief ray of the image light) differs by an angle β (e.g., 60°) from the direction of the image light incident from the retroreflector 2 (the direction of the chief ray of the image light). With this configuration, the optical system of FIG. 2B outputs the image light toward the outside of the transparent member 100 at a predetermined angle shown in the figure, forming the space-floating image 3, which is a real image. In the configuration of FIG. 2B , when a user views the space-floating image 3 from the direction of arrow A, the space-floating image 3 is perceived as a bright image. However, when another person views the space-floating image 3 from the direction of arrow B, the space-floating image 3 cannot be perceived as an image at all. This characteristic is extremely suitable for use in a system that displays images that require high security or highly confidential images that should be kept secret from people directly facing the user.

[0031] As described above, the optical system of FIG. 2B has a different configuration from the optical system of FIG. 2A, but can form a suitable floating image in space, similar to the optical system of FIG. 2A.

[0032] An absorptive polarizing plate may be provided on the surface of the transparent member 100 facing the polarization separation member 101B. This absorptive polarizing plate may transmit the polarized waves of the image light from the polarization separation member 101B and absorb the polarized waves that are 90° out of phase with the polarized waves of the image light from the polarization separation member 101B. In this way, the image light for forming the space-floating image 3 can be sufficiently transmitted while reducing the external light incident on the space-floating image 3 side of the transparent member 100 by approximately 50%. This makes it possible to reduce stray light in the optical system of FIG. 2B due to the external light incident on the space-floating image 3 side of the transparent member 100.

[0033] <Another configuration example 2 of the optical system of the space floating image display device> Another example of the configuration of the optical system of the space floating image display device will be explained using Fig. 2C. In Fig. 2C, components with the same reference numerals as Fig. 2B have the same functions and configurations as Fig. 2B. For the sake of simplicity, such components will not be described repeatedly.

[0034] The only difference between the optical system in Figure 2B and the optical system in Figure 2C is the angle at which the polarization separation member 101B is disposed relative to the image display surface of the display device 1 and the surface of the retroreflector 2. All other configurations are the same as those of the optical system in Figure 2B, so repeated explanations will be omitted. The polarization design of the optical system in Figure 2C is also the same as that of the optical system in Figure 2B, so repeated explanations will be omitted.

[0035] In the optical system of FIG. 2C , the polarization separator 101B is tilted at an angle α with respect to the image display surface of the display device 1 and the surface of the retroreflector 2. In FIG. 2C , the angle α is 45°. With this configuration, when the polarization separator 101B reflects, the angle β between the direction of propagation of the image light incident from the retroreflector 2 (the direction of the chief ray of the image light) and the direction of propagation of the image light reflected by the polarization separator 101B (the direction of the chief ray of the image light) is 90°. With this configuration, the image display surface of the display device 1 and the surface of the retroreflector 2 are perpendicular to the direction of propagation of the image light reflected by the polarization separator 101B, simplifying the angular relationships of the surfaces that make up the optical system. By arranging the surface of the transparent member 100 so that it is perpendicular to the direction of propagation of the image light reflected by the polarization separator 101B, the angular relationships of the surfaces that make up the optical system can be further simplified. In the configuration of Figure 2C, when a user views the floating image 3 from the direction of arrow A, the floating image 3 is perceived as a bright image. However, when another person views the floating image 3 from the direction of arrow B, the floating image 3 cannot be seen as an image at all. This characteristic is very suitable for use in a system that displays images that require high security or highly confidential images that should be kept secret from people directly facing the user.

[0036] As described above, the optical system of Fig. 2C has a different configuration from the optical systems of Fig. 2A and Fig. 2B, but can form a suitable floating image in space similar to the optical systems of Fig. 2A and Fig. 2B. In addition, the angles of the surfaces constituting the optical system can be made simpler.

[0037] An absorptive polarizer may be provided on the surface of the transparent member 100 facing the polarization separation member 101B. This absorptive polarizer may transmit the polarized waves of the image light from the polarization separation member 101B and absorb the polarized waves that are 90° out of phase with the polarized waves of the image light from the polarization separation member 101B. This allows the image light for forming the space-floating image 3 to be sufficiently transmitted while reducing the external light incident on the space-floating image 3 side of the transparent member 100 by approximately 50%. This allows the stray light in the optical system of FIG. 2C due to the external light incident on the space-floating image 3 side of the transparent member 100 to be reduced.

[0038] <Another configuration example 3 of the optical system of the space floating image display device> Another example of the configuration of the optical system of the space floating image display device will be described with reference to FIG. 2D. The optical system of FIG. 2D is an optical system that uses a retroreflector 5 that is different from the retroreflector 2 used in FIGS. 2A to 2C. Hereinafter, another example of the configuration 3 of the optical system will be described in more detail with reference to FIGS. 2D to 2I. In FIG. 2D, components that are given the same reference numerals as those in FIGS. 2A to 2C have the same functions and configurations as those in FIGS. 2A to 2C. For the sake of simplicity, such components will not be described repeatedly.

[0039] 2D is a diagram showing an example of the main components and retroreflection components of a space floating image display device according to one embodiment of the present invention. A display device 1 that emits image light is provided obliquely on a transparent member 100 such as glass. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates light.

[0040] A chief ray 9020 representing the light beam emitted from the display device 1 travels toward the retroreflector 5 and is incident on the retroreflector 5 at an incident angle α. The incident angle α may be, for example, 45°. However, the incident angle α is not limited to 45°, and may also be, for example, 45°±15°.

[0041] The retroreflector 5 is an optical element having the optical property of retroreflecting light rays in at least some directions. Furthermore, since the reflected light rays have the optical property of forming an image, the retroreflector 5 may also be referred to as an imaging optical element or an imaging optical plate.

[0042] 2E, 2F, etc., the principal ray 9020 travels in the z direction while being retroreflected in the x and y directions by the retroreflector 5. As a result, the reflected ray 9021 travels in a direction away from the retroreflector 5 along an optical path that is mirror-symmetrical with respect to the principal ray 9020 with the retroreflector 5 as the reference, passes through the transparent member 100, and forms the spatial floating image 3 as a real image on the imaging plane.

[0043] The light beam that forms the floating image 3 is a collection of light rays that converge from the retroreflector 5 to the optical image of the floating image 3, and these light rays continue to travel in a straight line even after passing through the optical image of the floating image 3. Therefore, the floating image 3 is an image with high directionality, unlike a diffuse image formed on a screen by a general projector or the like. Therefore, in the configuration of Figure 2, when a user views the floating image 3 from the direction of arrow A, the floating image 3 is perceived as a bright image. However, when another person views the floating image 3 from the direction of arrow B, the floating image 3 cannot be perceived as an image at all. This characteristic is suitable for use in a system that displays images that require high security or highly confidential images that should be kept secret from people directly facing the user.

[0044] An example of the configuration of the retroreflector 5 will be described using Figures 2E and 2F. The retroreflector 5 has a configuration in which multiple corner reflectors 9040 are arranged in an array on the surface of a transparent member. This may also be called a corner reflector array or a polyhedral reflector array. The specific configuration of the corner reflector 9040 will be described in detail using Figures 2G, 2H, and 2I. Light rays 9111, 9112, 9113, and 9114 emitted from a light source 9110 are reflected twice by two mirror surfaces 9041 and 9042 of the corner reflector 9040, becoming reflected light rays 9121, 9122, 9123, and 9124. This double reflection is retroreflection in the x and y directions, where the light returns to the same direction as the incident direction (traveling in a direction rotated 180 degrees). In the z direction, total reflection results in specular reflection, where the angle of incidence and the angle of reflection match.

[0045] That is, light rays 9111 to 9114 generate reflected light rays 9121 to 9124 on straight lines symmetrical in the z direction with respect to corner reflector 9040, and form aerial real image 9120. Note that light rays 9111 to 9114 emitted from light source 9110 are four light rays that represent the diffused light from light source 9110, and although the light rays incident on retroreflector 5 are not limited to these four light rays depending on the diffusion characteristics of light source 9110, all incident light rays cause similar reflections and form aerial real image 9120. Note that for ease of viewing the drawing, the position of light source 9110 and the position of aerial real image 9120 are shown shifted in the x direction, but in reality, the position of light source 9110 and the position of aerial real image 9120 in the x direction are the same and are overlapping when viewed from the z direction.

[0046] 2G, 2H, and 2I, the configuration and effects of the corner reflector 9040 that constitutes the retroreflector 5 will be described. The corner reflector 9040 is a rectangular parallelepiped with only two specific faces being mirror surfaces 9041 and 9042, and the other four faces being made of transparent materials. The retroreflector 5 has a configuration in which these corner reflectors 9040 are arrayed so that corresponding mirror surfaces face in the same direction.

[0047] When viewed from the top (+z direction), a light ray 9111 emitted from a light source 9110 enters the mirror surface 9041 (or the mirror surface 9042) at a specific angle of incidence, is totally reflected at a reflection point 9130, and then is totally reflected again at a reflection point 9132 on the mirror surface 9042 (or the mirror surface 9041).

[0048] If the angle of incidence of light ray 9111 with respect to mirror surface 9041 (or mirror surface 9042) is θ, then the angle of incidence of first reflected light ray 9131 reflected by mirror surface 9041 (or mirror surface 9042) with respect to mirror surface 9042 (or mirror surface 9041) can be expressed as 90°-θ. Therefore, with respect to light ray 9111, second reflected light ray 9121 is rotated by 2θ after the first reflection and by 2×(90°-θ) after the second reflection, resulting in a total reversal optical path of 180°. On the other hand, when viewed from the side (the direction halfway between -x and -y), total reflection in the z direction occurs only once. Therefore, if the angle of incidence with respect to mirror surface 9041 or mirror surface 9042 is φ, then reflected light ray 9121 is rotated by 2×φ after one reflection with respect to light ray 9111.

[0049] As described above, light rays incident on the corner reflector 9040 undergo retroreflection, which creates an inverted optical path in the x and y directions, and specular reflection due to total reflection in the z direction. Considering the retroreflector 5, similar reflections occur in each optical path, so that an image is formed at a point symmetrical with respect to the z axis direction by an inverted optical path that is convergent in the x and y directions.

[0050] 2A to 2C, the retroreflector 2 has retroreflection properties in three axes. As a result, when a diffusive incident light beam is incident on the retroreflector 2, a convergent reflected light beam travels toward the side of the retroreflector 2 where the light source of the incident light beam is located. The convergent reflected light beam forms an image in the air, forming a space-floating image 3. The traveling direction of the chief ray of the convergent reflected light beam reflected from the retroreflector 2 is opposite to the traveling direction of the chief ray of the diffusive incident light beam incident on the retroreflector 2.

[0051] In contrast, in the optical system of Fig. 2D, the retroreflector 5 has retroreflection properties in two axial directions and specular reflection in the other axial direction. As a result, when a diffusive incident light beam is incident on the retroreflector 5, the convergent reflected light beam is reflected by the corner reflector array and travels in the direction opposite to the side of the retroreflector 5 where the light source of the incident light is located. The convergent reflected light beam forms an image in the air and forms the space floating image 3.

[0052] The traveling direction of the chief ray of the convergent reflected light beam reflected by the corner reflector array of the retroreflector 5 is not the opposite direction to the traveling direction of the chief ray of the diffusive incident light beam that is incident on the retroreflector 5. The normal direction component of the plate-shaped surface of the retroreflector 5 in the traveling direction of the chief ray of the diffusive incident light beam that is incident on the retroreflector 5 and the normal direction component of the plate-shaped surface of the retroreflector 5 in the traveling direction of the chief ray after being reflected by the retroreflector 5 and becoming a convergent reflected light beam continue to travel in a straight line, unchanged before and after reflection by the corner reflector array.

[0053] That is, the diffusive incident light beam is converted into a convergent reflected light beam by reflection on the retroreflector 5, but in the normal direction to the plate-shaped surface of the retroreflector 5, the light beam travels as if passing through the retroreflector 5. Here, the diffusive incident light beam incident on the retroreflector 5 and the convergent reflected light beam emerging from the retroreflector 5 are in a geometrically symmetrical relationship with respect to the plate-shaped surface of the retroreflector 5.

[0054] The resolution of the space-floating image formed by the light beams from the display device 1 depends not only on the resolution of the liquid crystal display panel 11 but also on the diameter D and pitch P (not shown) of the retroreflective portion of the retroreflector 5 shown in Figures 2E and 2F. For example, when using a 7-inch WUXGA (1920 x 1200 pixels) liquid crystal display panel, even if one pixel (one triplet) is approximately 80 μm, if the diameter D of the retroreflective portion is 240 μm and the pitch P is 300 μm, one pixel of the space-floating image will be equivalent to 300 μm. As a result, the effective resolution of the space-floating image will be reduced to about one-third.

[0055] Therefore, in order to make the resolution of the spatial floating image equivalent to that of the display device 1, it is desirable to make the diameter D and pitch P of the retroreflective portion close to one pixel of the liquid crystal display panel. On the other hand, to suppress the occurrence of moire caused by the retroreflective plate and the pixels of the liquid crystal display panel, it is advisable to design the pitch ratio of each to be a different integer multiple of one pixel. Also, it is advisable to arrange the shape so that none of the sides of the retroreflective portion overlaps with any of the sides of one pixel of the liquid crystal display panel.

[0056] The shape of the retroreflector (imaging optical plate) according to this embodiment is not limited to the above example. It may have various shapes that achieve retroreflection. Specifically, it may be a variety of cubic corner bodies, a corner reflector array, a slit mirror array, a dihedral corner reflector array, a polyhedral reflector array, or a shape in which a combination of these reflective surfaces is periodically arranged. Alternatively, a capsule lens-type retroreflector element with periodically arranged glass beads may be provided on the surface of the retroreflector according to this embodiment. The detailed configuration of these retroreflectors can be achieved using existing technology, so a detailed description will be omitted. Specifically, the technology disclosed in JP 2017-33005 A, JP 2019-133110 A, JP 2017-67933 A, WO 2009 / 131128 A, etc. may be used.

[0057] 2D, the image light emitted from the display device 1 may be in any polarization state, either S-polarized or P-polarized.

[0058] As explained above, the optical system of Figure 2D is an optical system that uses a retroreflector different from the optical systems of Figures 2A to 2C, but it can form a more suitable floating image in space, similar to the optical systems of Figures 2A to 2C.

[0059] According to the optical systems of FIGS. 2A, 2B, 2C, and 2D described above, it is possible to provide brighter, higher quality floating images in space.

[0060] <<Block diagram of the internal configuration of the space floating image display device>> Next, a description will be given of a block diagram of the internal configuration of the space-floating image display device 1000. Fig. 3 is a block diagram showing an example of the internal configuration of the space-floating image display device 1000.

[0061] The space-floating image display device 1000 includes a retroreflection unit 1101, an image display unit 1102, a light guide 1104, a light source 1105, a power supply 1106, an external power supply input interface 1111, an operation input unit 1107, a nonvolatile memory 1108, a memory 1109, a control unit 1110, a video signal input unit 1131, an audio signal input unit 1133, a communication unit 1132, an aerial operation detection sensor 1351, an aerial operation detection unit 1350, an audio output unit 1140, a microphone 1139, an image control unit 1160, a storage unit 1170, an imaging unit 1180, etc. In addition, the space-floating image display device 1000 may also include a removable media interface 1134, an attitude sensor 1113, a transmissive self-luminous image display device 1650, a second display device 1680, or a secondary battery 1112.

[0062] Each component of the space floating image display device 1000 is disposed in a housing 1190. The imaging unit 1180 and the mid-air operation detection sensor 1351 shown in FIG.

[0063] The retroreflector 1101 in Fig. 3 corresponds to the retroreflector 2 in Figs. 2A, 2B, and 2C. The retroreflector 1101 retroreflects light modulated by the image display unit 1102. Of the light reflected from the retroreflector 1101, the light output to the outside of the space-floating image display device 1000 forms the space-floating image 3. When the optical system in Fig. 2D is applied, the retroreflector 1101 corresponds to the retroreflector 5 in Fig. 2D.

[0064] 3 corresponds to the liquid crystal display panel 11 in FIGS. 2A, 2B, and 2C. The light source 1105 in FIG. 3 corresponds to the light source device 13 in FIGS. 2A, 2B, and 2C. The image display unit 1102, the light guide 1104, and the light source 1105 in FIG. 3 correspond to the display device 1 in FIGS. 2A, 2B, and 2C.

[0065] The video display unit 1102 is a display unit that generates a video by modulating transmitted light based on a video signal input under the control of a video control unit 1160 (described later). The video display unit 1102 (the above-described liquid crystal display panel 11) may be, for example, a transmissive liquid crystal panel, but is not limited to this. Alternatively, the video display unit 1102 may be, for example, a reflective liquid crystal panel that modulates reflected light, a DMD (Digital Micromirror Device: registered trademark) panel, or the like.

[0066] The light source 1105 generates light for the image display unit 1102 and is a solid-state light source such as an LED (Light Emitting Diode) or a laser light source. The power source 1106 converts AC current input from the outside via the external power input interface 1111 into DC current and supplies power to the light source 1105. The power source 1106 also supplies the necessary DC current to each unit within the space-floating image display device 1000. The secondary battery 1112 stores the power supplied from the power source 1106. The secondary battery 1112 also supplies power to the light source 1105 and other components that require power via the external power input interface 1111 when power is not supplied from the outside. In other words, when the space-floating image display device 1000 is equipped with the secondary battery 1112, the user can use the space-floating image display device 1000 even when power is not supplied from the outside.

[0067] The light guide 1104 guides light generated by the light source 1105 and irradiates it onto the video display unit 1102. The combination of the light guide 1104 and the light source 1105 can also be called a backlight for the video display unit 1102. The light guide 1104 may be configured mainly using glass. The light guide 1104 may be configured mainly using plastic. The light guide 1104 may be configured using a mirror. Various methods are possible for combining the light guide 1104 and the light source 1105. Specific configuration examples of the combination of the light guide 1104 and the light source 1105 will be described in detail later.

[0068] The aerial operation detection sensor 1351 is a sensor that detects an operation of the floating in space image 3 by an operating object such as a user's finger. The aerial operation detection sensor 1351 senses, for example, an area that overlaps with the entire display area of ​​the floating in space image 3. Note that the aerial operation detection sensor 1351 may only sense an area that overlaps with at least a portion of the display area of ​​the floating in space image 3.

[0069] Specific examples of the aerial operation detection sensor 1351 include a distance sensor that uses invisible light such as infrared light, an invisible laser, ultrasonic waves, etc. The aerial operation detection sensor 1351 may also be configured to detect coordinates on a two-dimensional plane by combining multiple sensors. The aerial operation detection sensor 1351 may also be configured with a ToF (Time of Flight) LiDAR (Light Detection and Ranging) or an image sensor.

[0070] The mid-air operation detection sensor 1351 only needs to be capable of sensing to detect touch operations with the user's finger on objects displayed as the floating-in-space image 3. Such sensing can be performed using existing technology.

[0071] The aerial operation detection unit 1350 acquires a sensing signal from the aerial operation detection sensor 1351, and, based on the sensing signal, determines whether or not the user's finger has made contact with an object in the floating in space image 3, and calculates the position (contact position) where the user's finger has made contact with the object. The aerial operation detection unit 1350 is configured, for example, by a circuit such as an FPGA (Field Programmable Gate Array). Furthermore, some of the functions of the aerial operation detection unit 1350 may be realized by software, for example, by a program for detecting aerial operation executed by the control unit 1110 or the image control unit 1160. The aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be configured as an integrated unit. The aerial operation detection unit 1350 and the control unit 1110 or the image control unit 1160 may be configured as an integrated unit.

[0072] The aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be configured to be built into the space-floating image display device 1000, or may be provided externally as a separate entity from the space-floating image display device 1000. When provided as a separate entity from the space-floating image display device 1000, the aerial operation detection sensor 1351 and the aerial operation detection unit 1350 are configured to be able to transmit information and signals to the space-floating image display device 1000 via a wired or wireless communication connection path or a video signal transmission path. This makes it possible to build a system in which the space-floating image display device 1000, which does not have aerial operation detection function, is used as the main body, and only the aerial operation detection function can be added as an option.

[0073] Also, the aerial operation detection sensor 1351 may be a separate unit, and the aerial operation detection unit 1350 may be built into the space-floating image display device 1000. In cases where it is desired to more freely arrange the aerial operation detection sensor 1351 relative to the installation position of the space-floating image display device 1000, there is an advantage to a configuration in which only the aerial operation detection sensor 1351 is a separate unit.

[0074] The imaging unit 1180 is a camera with an image sensor, and captures images of the space near the space-floating image 3 and / or the face, arms, fingers, etc. of the user 230. A plurality of imaging units 1180 may be provided. For example, the imaging unit 1180 may be provided as a stereo camera. By using a plurality of imaging units 1180, or by using an imaging unit with a depth sensor, the mid-air operation detection unit 1350 can be assisted in detecting the touch operation of the space-floating image 3 by the user 230. The imaging unit 1180 may be provided separately from the space-floating image display device 1000. When the imaging unit 1180 is provided separately from the space-floating image display device 1000, it is sufficient to configure it so that an imaging signal can be transmitted to the space-floating image display device 1000 via a wired or wireless communication connection path, etc.

[0075] For example, if the aerial operation detection sensor 1351 is configured as an object intrusion sensor that targets a plane (intrusion detection plane) including the display surface (display range) of the spatial floating image 3 and detects whether or not an object has intruded into this intrusion detection plane, the aerial operation detection sensor 1351 may not be able to detect information such as how far an object (e.g., a user's finger) that has not intruded into the intrusion detection plane is from the intrusion detection plane, or how close the object is to the intrusion detection plane.

[0076] In such a case, the distance between the object and the intrusion detection plane (space floating image 3) can be calculated by using information such as object depth calculation information based on captured images from the multiple imaging units 1180 and object depth information from a depth sensor. Then, various information such as this depth calculation information, depth information, and distance between the object and the intrusion detection plane is used for various display controls for the space floating image 3.

[0077] Furthermore, without using the aerial operation detection sensor 1351, the aerial operation detection unit 1350 may detect a touch operation on the floating-in-space image 3 by the user 230 based on the captured image of the imaging unit 1180. In this case, the imaging unit 1180 may be referred to as an aerial operation detection sensor.

[0078] Furthermore, the imaging unit 1180 may capture an image of the face of the user operating the space-floating image 3, and the control unit 1110 or the like may perform a process to identify the user. Furthermore, in order to determine whether or not there is another person standing around or behind the user operating the space-floating image 3 and peeking at the user's operation of the space-floating image 3, the imaging unit 1180 may capture an image of the user operating the space-floating image 3 and a range including the user's surrounding area.

[0079] The operation input unit 1107 is, for example, an operation button, a signal receiving unit such as a remote controller, or an infrared light receiving unit, and inputs signals for operations different from the user's air operation (touch operation). Apart from the above-mentioned user who touches the space floating image 3, the operation input unit 1107 may also be used by, for example, an administrator to operate the space floating image display device 1000.

[0080] The video signal input unit 1131 is connected to an external video output device and inputs video data (video signals). The video signal input unit 1131 can be implemented using various digital video input interfaces. For example, it may be configured with a video input interface conforming to the HDMI (registered trademark) (High-Definition Multimedia Interface) standard, a video input interface conforming to the DVI (Digital Visual Interface) standard, or a video input interface conforming to the DisplayPort standard. Alternatively, an analog video input interface such as analog RGB or composite video may be provided.

[0081] The audio signal input unit 1133 is connected to an external audio output device and inputs audio data (audio signals). The audio signal input unit 1133 may be configured as an audio input interface conforming to the HDMI standard, an optical digital terminal interface, a coaxial digital terminal interface, or the like. In the case of an interface conforming to the HDMI standard, the video signal input unit 1131 and the audio signal input unit 1133 may be configured as an interface in which a terminal and a cable are integrated.

[0082] The audio output unit 1140 can output audio based on audio data input to the audio signal input unit 1133. The audio output unit 1140 may be configured with a speaker 1140. The audio output unit 1140 may also include a section that performs voice synthesis processing, etc. The audio output unit 1140 may also output built-in operation sounds and error warning sounds. Alternatively, the audio output unit 1140 may be configured to output a digital signal to an external device, like the Audio Return Channel function defined in the HDMI standard.

[0083] The audio input unit 1139 may be configured with a microphone 1139. The microphone 1139 is a microphone that collects sounds around the space-floating image display device 1000, converts them into signals, and generates audio signals. The microphone may record a person's voice, such as a user's voice, and the control unit 1110 or the like performs voice recognition processing on the generated audio signal to obtain text information from the audio signal. The audio input unit 1139 may be provided with a part that performs voice recognition processing or the like. Note that the audio output unit 1140, the audio input unit 1139, etc. may be connected as external devices to the space-floating image display device 1000.

[0084] The nonvolatile memory 1108 stores various data used by the space-floating image display device 1000. The data stored in the nonvolatile memory 1108 includes, for example, data for various operations to be displayed on the space-floating image 3, display icons, data of objects for the user to operate, layout information, etc. The memory 1109 stores image data to be displayed as the space-floating image 3, data for controlling the device, etc.

[0085] The control unit 1110 includes a processor and controls the operation of each connected unit. The control unit 1110 may also work in cooperation with a program stored in the memory 1109 to perform calculations based on information acquired from each unit in the space floating image display device 1000.

[0086] The communication unit 1132 communicates with external devices, external servers, etc. via a wired or wireless communication interface. If the communication unit 1132 has a wired communication interface, the wired communication interface may be configured, for example, as an Ethernet LAN interface. If the communication unit 1132 has a wireless communication interface, the interface may be configured, for example, as a Wi-Fi communication interface, a Bluetooth communication interface, or a mobile communication interface such as 4G or 5G. Various types of data, such as video data, image data, and audio data, are transmitted and received through communication via the communication unit 1132.

[0087] The removable media interface 1134 is an interface for connecting a removable recording medium (removable media). The removable recording medium (removable media) may be composed of a semiconductor device memory such as a solid state drive (SSD), a magnetic recording medium recording device such as a hard disk drive (HDD), or an optical recording medium such as an optical disk. The removable media interface 1134 can read various information such as video data, image data, and audio data recorded on the removable recording medium. The video data, image data, etc. recorded on the removable recording medium are output as the floating image 3 via the video display unit 1102 and the retroreflection unit 1101.

[0088] The storage unit 1170 is a storage device that records various types of information such as video data, image data, audio data, etc. The storage unit 1170 may be configured with a magnetic recording medium recording device such as a hard disk drive (HDD), or a semiconductor element memory such as a solid state drive (SSD). For example, various types of information such as video data, image data, audio data, etc. may be recorded in advance in the storage unit 1170 at the time of product shipment. Furthermore, the storage unit 1170 may record various types of information such as video data, image data, audio data, etc. acquired from an external device, an external server, etc. via the communication unit 1132.

[0089] The video data, image data, etc. recorded in the storage unit 1170 are output as the space floating image 3 via the video display unit 1102 and the retroreflection unit 1101 based on processing by the video control unit 1160. Video data, image data, etc. of display icons, objects for user operation, etc. displayed as the space floating image 3 are also recorded in the storage unit 1170. Layout information of the display icons, objects, etc. displayed as the space floating image 3, and various metadata information related to the objects, etc. are also recorded in the storage unit 1170.

[0090] The audio data recorded in the storage unit 1170 is output as audio from the audio output unit 1140, for example.

[0091] The video control unit 1160 performs various controls related to the video signal input to the video display unit 1102. Based on the video signal (video data), the video control unit 1160 creates a video signal (display data) for displaying a video on the video display unit 1102 (for example, the liquid crystal display panel 11 of the display device 1 described above), and supplies the video signal to the video display unit 1102. The video control unit 1160 may be referred to as a video processing circuit, and may be configured with hardware such as an ASIC, FPGA, or video processor. The video control unit 1160 may also be referred to as a video processing unit or an image processing unit. The video control unit 1160 performs video switching control, such as determining which video signal to input to the video display unit 1102, between the video signal to be stored in the memory 1109 and the video signal (video data) input to the video signal input unit 1131.

[0092] Note that the control unit 1110 may perform the same processing as the video control unit 1160, in which case the control unit 1110 may be referred to as a video processing unit, etc. At least one of the control unit 1110, the video control unit 1160, the aerial operation detection unit 1360, etc. may perform unique control processing, in which case the control unit 1110, the video control unit 1160, the aerial operation detection unit 1360, etc. may be referred to as a video processing unit.

[0093] In addition, the video control unit 1160 may generate a superimposed video signal by superimposing the video signal to be stored in the memory 1109 and the video signal input from the video signal input unit 1131, and input the superimposed video signal to the video display unit 1102, thereby performing control to form the composite video as the floating-in-space video 3.

[0094] Furthermore, the video control unit 1160 may control image processing of the video signal input from the video signal input unit 1131, the video signal to be stored in the memory 1109, etc. Examples of image processing include scaling processing to enlarge, reduce, deform, etc. the image, brightness adjustment processing to change the brightness, contrast adjustment processing to change the contrast curve of the image, and Retinex processing to decompose the image into light components and change the weighting of each component.

[0095] Furthermore, the video control unit 1160 may perform special effect video processing or the like to assist the user's mid-air operation (touch operation) on the video signal input to the video display unit 1102. The special effect video processing is performed, for example, based on the detection result of the user's touch operation by the mid-air operation detection unit 1350, or on the image of the user captured by the imaging unit 1180. Furthermore, the video control unit 1160 or the like may perform audio control processing when audio is output from the audio output unit 1140 simultaneously with the floating-in-space video 3. An audio control unit for this audio control processing may be provided separately from the video control unit 1160.

[0096] The attitude sensor 1113 is a sensor configured by a gravity sensor or an acceleration sensor, or a combination of these, and can detect the attitude in which the space-floating image display device 1000 is installed. Based on the attitude detection result of the attitude sensor 1113, the control unit 1110 may control the operation of each connected unit. For example, when an undesirable attitude in the user's usage state is detected, the control unit 1110 may perform control such that the image being displayed on the image display unit 1102 is stopped and an error message is displayed to the user. Alternatively, when the attitude sensor 1113 detects a change in the installation attitude of the space-floating image display device 1000, the control unit 1110 may perform control such that the display direction of the image being displayed on the image display unit 1102 is rotated.

[0097] As explained above, various functions are installed in the space-floating image display device 1000. However, the space-floating image display device 1000 does not need to have all of these functions, and any configuration is acceptable as long as it has the function of forming the space-floating image 3.

[0098] <Configuration example of a space floating image display device> Next, a configuration example of the space-floating image display device will be described. The layout of the components of the space-floating image display device according to this embodiment can be various depending on the usage form. Below, the layouts of each of Figs. 4A to 4P will be described. In addition, in each example of Figs. 4A to 4P, the thick lines surrounding the components (display device 1, etc.) of the space-floating image display device 1000 show an example of the housing structure (housing 1190 in Fig. 3) of the space-floating image display device 1000.

[0099] FIG. 4A is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4A is equipped with an optical system corresponding to the optical system of FIG. 2A. The space-floating image display device 1000 shown in FIG. 4A is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4A, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. When the mid-air operation detection sensor 1351 is installed as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230. Note that the x direction is the left-right direction as seen from the user, the y direction is the front-back direction (depth direction) as seen from the user, and the z direction is the up-down direction (vertical direction). Hereinafter, the definitions of the x direction, y direction, and z direction are the same in each drawing of FIG. 4, so repeated explanations will be omitted.

[0100] FIG. 4B is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4B is equipped with an optical system corresponding to the optical system of FIG. 2A. The space-floating image display device 1000 shown in FIG. 4B is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (toward the user 230). That is, in FIG. 4B, the space-floating image display device is installed with the transparent member 100 facing the front of the device (toward the user 230). The space-floating image 3 is formed on the user 230 side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230. 4B, the mid-air operation detection sensor 1351 can use the reflection of sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, the reflectivity of the nail is higher than that of the finger pad, so this configuration can improve the accuracy of touch detection.

[0101] FIG. 4C is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4C is equipped with an optical system corresponding to the optical system of FIG. 2B. The space-floating image display device 1000 shown in FIG. 4C is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4C, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the user's 230 finger.

[0102] FIG. 4D is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4D is equipped with an optical system corresponding to the optical system of FIG. 2B. The space-floating image display device 1000 shown in FIG. 4D is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (toward the user 230). That is, in FIG. 4D, the space-floating image display device 1000 is installed with the transparent member 100 facing the front of the device (toward the user 230). The space-floating image 3 is formed on the user 230 side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the user 230's finger. 4D, the mid-air operation detection sensor 1351 can use the reflection of sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, the reflectivity of the nail is higher than that of the finger pad, so this configuration can improve the accuracy of touch detection.

[0103] FIG. 4E is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4E is equipped with an optical system corresponding to the optical system of FIG. 2C. The space-floating image display device 1000 shown in FIG. 4E is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4E, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels directly upward. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230.

[0104] FIG. 4F is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4F is equipped with an optical system corresponding to the optical system of FIG. 2C. The space-floating image display device 1000 shown in FIG. 4F is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (toward the user 230). That is, in FIG. 4F, the space-floating image display device 1000 is installed with the transparent member 100 facing the front of the device (toward the user 230). The space-floating image 3 is formed on the user 230 side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels toward the user. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the user 230's finger.

[0105] FIG. 4G is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4G is equipped with an optical system corresponding to the optical system shown in FIG. 2C. In the optical systems of the space-floating image display devices shown in FIGS. 4A to 4F, the central optical path of the image light emitted from the display device 1 was on the yz plane. That is, in the optical systems of the space-floating image display devices shown in FIGS. 4A to 4F, the image light traveled in the front-to-back and up-to-down directions as seen from the user. In contrast, in the optical system of the space-floating image display device shown in FIG. 4G, the central optical path of the image light emitted from the display device 1 is on the xy plane. That is, in the optical system of the space-floating image display device shown in FIG. 4G, the image light travels in the left-to-right and front-to-back directions as seen from the user. The space-floating image display device 1000 shown in FIG. 4G is installed so that the surface on which the space-floating image 3 is formed faces the front of the device (toward the user 230). That is, in Fig. 4G, the space-floating image display device 1000 has the transparent member 100 installed on the front side of the device (toward the user 230). The space-floating image 3 is formed on the user side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels towards the user. If the mid-air operation detection sensor 1351 is installed as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230.

[0106] FIG. 4H is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4H differs from the space-floating image display device of FIG. 4G in that it has a window with a transparent plate 100B made of glass or plastic on the back of the device (opposite the position where the user 230 views the space-floating image 3, i.e., opposite the traveling direction of the image light of the space-floating image 3 toward the user 230). The rest of the configuration is the same as that of the space-floating image display device of FIG. 4G, so repeated explanations will be omitted. The space-floating image display device 1000 of FIG. 4H has a window with a transparent plate 100B on the opposite side of the traveling direction of the image light of the space-floating image 3 from the space-floating image 3. Therefore, when the user 230 views the space-floating image 3, they can recognize the scenery behind the space-floating image display device 1000 as the background of the space-floating image 3. Therefore, the user 230 can perceive the space floating image 3 as floating in the air in front of the scenery behind the space floating image display device 1000. This can further emphasize the feeling of the space floating image 3 floating in the air.

[0107] Depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, a portion of the image light output from the display device 1 may be reflected by the polarization separation member 101B and head toward the transparent plate 100B. Depending on the coating performance of the surface of the transparent plate 100B, this light may be reflected again by the surface of the transparent plate 100B and may be visually recognized by the user 230 as stray light. Therefore, in order to prevent this stray light, the transparent plate 100B may not be provided at the window on the back of the space-floating image display device 1000.

[0108] Fig. 4I is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4I is different from the space-floating image display device of Fig. 4H in that a light-blocking door 1410 is provided in the window of the transparent plate 100B located on the back of the device (opposite the position where the user 230 views the space-floating image 3). The other configurations are the same as those of the space-floating image display device of Fig. 4H, so repeated explanations will be omitted.

[0109] The opening and closing door 1410 of the space-floating image display device 1000 in FIG. 4I has, for example, a light blocking plate, and is equipped with a mechanism for moving (sliding), rotating, or attaching / detaching the light blocking plate, thereby switching between an open state and a light blocking state for the window (rear window) of the transparent plate 100B located at the back of the space-floating image display device 1000. The movement (sliding) and rotation of the light blocking plate by the opening and closing door 1410 may be electrically driven by a motor (not shown). The motor may be controlled by the control unit 1110 in FIG. 3. Note that the example in FIG. 4I discloses an example in which the opening and closing door 1410 has two light blocking plates. In contrast, the opening and closing door 1410 may have only one light blocking plate.

[0110] For example, when the view seen through the window of the transparent plate 100B of the space-floating image display device 1000 is outdoors, the brightness of sunlight varies depending on the weather. When the outdoor sunlight is strong, the background of the space-floating image 3 may become too bright, reducing the user 230's visibility of the space-floating image 3. In such a case, by moving (sliding), rotating, or attaching the light blocking plate of the opening / closing door 1410 to block the light from the rear window, the background of the space-floating image 3 becomes dark, thereby relatively increasing the visibility of the space-floating image 3. Such a blocking operation by the light blocking plate of the opening / closing door 1410 may be performed directly by the force of the user 230's hand. In response to an operation input via the operation input unit 1107 of FIG. 3, the control unit 1110 may control a motor (not shown) to perform the blocking operation by the light blocking plate of the opening / closing door 1410.

[0111] An illuminance sensor may be provided on the rear side (opposite the user 230) of the space-floating image display device 1000, such as near the rear window, to measure the brightness of the space beyond the rear window. In this case, the control unit 1110 of Fig. 3 may control a motor (not shown) to perform the opening and closing operation of the light blocking plate of the opening and closing door 1410 according to the detection result of the illuminance sensor. By controlling the opening and closing operation of the light blocking plate of the opening and closing door 1410 in this way, it becomes possible to more suitably maintain the visibility of the space-floating image 3, even if the user 230 does not manually open and close the light blocking plate of the opening and closing door 1410.

[0112] Furthermore, the light blocking plate by the opening and closing door 1410 may be manually detachable. Depending on the intended use and installation environment of the space floating image display device 1000, the user can select whether to leave the rear window open or in a light blocking state. If it is planned to use the rear window in a light blocking state for a long period of time, the detachable light blocking plate can be fixed in the light blocking state. Also, if it is planned to use the rear window in an open state for a long period of time, it can be used with the detachable light blocking plate removed. The light blocking plate may be attached and detached using screws, a hook structure, or a fitting structure.

[0113] Even in the example of the space-floating image display device 1000 shown in FIG. 4I, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separator 101B, a portion of the image light output from the display device 1 may be reflected by the polarization separator 101B and directed toward the transparent plate 100B. Depending on the coating performance of the surface of the transparent plate 100B, this light may be reflected again by the surface of the transparent plate 100B and be visible to the user 230 as stray light. Therefore, to prevent this stray light, the window on the back of the space-floating image display device 1000 may not be provided with the transparent plate 100B. The above-described opening / closing door 1410 may be provided in a window that does not have the transparent plate 100B. To prevent this stray light, it is desirable that the inner surface of the housing of the light-shielding plate of the above-described opening / closing door 1410 have a coating or material with low light reflectance.

[0114] FIG. 4J is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4J differs from the space-floating image display device of FIG. 4H in that instead of the transparent plate 100B made of glass or plastic being placed on the rear side window, an electronically controlled transmittance variable device 1620 is placed. The other components are the same as those of the space-floating image display device of FIG. 4H, so repeated explanations will be omitted. An example of the electronically controlled transmittance variable device 1620 is a liquid crystal shutter. Although the electronically controlled transmittance variable device 1620 is not shown in FIG. 3, if it is provided, it may be configured as one component of the space-floating image display device 1000 of FIG. 3 and connected to other processing units such as the control unit 1110.

[0115] The liquid crystal shutter can control the light transmittance by controlling the voltage of the liquid crystal element sandwiched between two polarizing plates. Therefore, if the liquid crystal shutter is controlled to increase the transmittance, the scenery through the rear window can be seen through the background of the floating image 3. On the other hand, if the liquid crystal shutter is controlled to decrease the transmittance, the scenery through the rear window can be hidden as the background of the floating image 3.

[0116] Furthermore, because the liquid crystal shutter can control halftones, it can also be set to a state of transmittance of 50% or the like. For example, the control unit 1110 can control the transmittance of the electronically controlled transmittance variable device 1620 in response to an operation input via the operation input unit 1107 in Fig. 3. With this configuration, in cases where a viewer wants to see the scenery through the rear window as the background of the Space Floating Image 3, but the scenery through the rear window as the background is too bright and reduces the visibility of the Space Floating Image 3, it is possible to adjust the transmittance of the electronically controlled transmittance variable device 1620 and thereby adjust the visibility of the Space Floating Image 3.

[0117] In addition, an illuminance sensor may be provided on the back side (opposite the user 230) of the space-floating image display device 1000, such as near the rear window, to measure the brightness of the space beyond the rear window. In this case, the control unit 1110 in Fig. 3 controls the transmittance of the electronically controlled transmittance variable device 1620 according to the detection result of the illuminance sensor. In this way, even if the user 230 does not perform an operation input via the operation input unit 1107 in Fig. 3, the transmittance of the electronically controlled transmittance variable device 1620 can be adjusted according to the brightness of the space beyond the rear window, making it possible to more suitably maintain the visibility of the space-floating image 3.

[0118] In the above example, a liquid crystal shutter has been described as an example of the electronically controlled variable transmittance device 1620. However, electronic paper may be used as another example of the electronically controlled variable transmittance device 1620. The same effects as those described above can be obtained when electronic paper is used. Furthermore, electronic paper consumes very little power to maintain a halftone state. Therefore, a space floating image display device with lower power consumption can be realized compared to when a liquid crystal shutter is used.

[0119] Fig. 4K is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4K differs from the space-floating image display device of Fig. 4G in that it has a transmissive self-luminous image display device 1650 instead of the transparent member 100. The other configurations are the same as those of the space-floating image display device of Fig. 4G, so repeated explanations will be omitted.

[0120] In the space-floating image display device 1000 of FIG. 4K, a light beam of an image passes through the display surface of the transmissive self-luminous image display device 1650, and then a space-floating image 3 is formed outside the space-floating image display device 1000. That is, when an image is displayed on the transmissive self-luminous image display device 1650, which is a two-dimensional flat display, the space-floating image 3 can be displayed as a pop-up image further in front of the image displayed on the transmissive self-luminous image display device 1650. In this case, the user 230 can simultaneously view two images at different depth positions. The transmissive self-luminous image display device 1650 may be configured using existing technology such as a transmissive organic EL panel disclosed in, for example, Japanese Patent Application Laid-Open No. 2014-216761. When the transmissive self-luminous image display device 1650 is provided, it may be configured to be connected to other processing units such as the control unit 1110 as a component of the space-floating image display device 1000 of FIG. 3.

[0121] Here, if the transmissive self-luminous video display device 1650 displays both the background and an object such as a character, and then displays only the object such as the character moving to the front of the floating video image 3, it is possible to provide the user 230 with a more effective video experience with a surprise effect.

[0122] Furthermore, if the inside of the space-floating image display device 1000 (housing 1190) is kept in a light-blocking state, the background of the transmissive self-luminous image display device 1650 will be sufficiently dark. Therefore, when no image is displayed on the display device 1 or the light source of the display device 1 is turned off and an image is displayed only on the transmissive self-luminous image display device 1650, the transmissive self-luminous image display device 1650 appears to the user 230 as a normal two-dimensional flat display rather than a transmissive display. Note that, since the space-floating image 3 in the embodiment of the present invention is displayed as a real optical image in a space without a screen, if the light source of the display device 1 is turned off, the intended display position of the space-floating image 3 becomes empty space. Therefore, when the transmissive self-luminous image display device 1650 is used to display an image as if it were a normal two-dimensional flat display, characters, objects, etc. can be suddenly displayed in the air as the space-floating image 3, thereby providing the user 230 with a more effective surprise video experience.

[0123] Note that the darker the interior of the space-floating image display device 1000, the more the transmissive self-luminous image display device 1650 appears like a two-dimensional flat display. Therefore, an absorptive polarizer (not shown) that transmits the polarized waves of the image light reflected by the polarization separation member 101B and absorbs polarized waves that are 90° out of phase with the polarized waves may be provided on the surface of the transmissive self-luminous image display device 1650 facing the interior of the space-floating image display device 1000 (the surface where the image light reflected by the polarization separation member 101B enters the transmissive self-luminous image display device 1650, i.e., the surface of the transmissive self-luminous image display device 1650 opposite the space-floating image 3). This does not have a significant effect on the image light that forms the space-floating image 3, but it can significantly reduce the light that enters the interior of the space-floating image display device 1000 from the outside through the transmissive self-luminous image display device 1650, making the interior of the space-floating image display device 1000 darker, which is preferable.

[0124] 4L is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4L is a modified example of the space-floating image display device of FIG. 4K. The orientation of the components in the space-floating image display device 1000 is different from that of the space-floating image display device of FIG. 4K, and is closer to the arrangement of the space-floating image display device of FIG. 4F. The functions and operations of each component are the same as those of the space-floating image display device of FIG. 4K, so repeated explanations will be omitted.

[0125] In the space-floating image display device of FIG. 4L, after the luminous flux of image light passes through the transmissive self-luminous image display device 1650, a space-floating image 3 is formed on the user 230 side of the transmissive self-luminous image display device 1650.

[0126] In both the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L, the space-floating image 3 is displayed superimposed on the image of the transmissive self-luminous image display device 1650 as seen by the user 230. Here, the position of the space-floating image 3 and the position of the image of the transmissive self-luminous image display device 1650 are configured to have a difference in the depth direction. Therefore, when the user 230 moves his / her head (position of viewpoint), the user can recognize the depth of the two images due to parallax. Therefore, by displaying two images at different depth positions, a three-dimensional image experience can be more suitably provided to the user with the naked eye, without the need for stereoscopic glasses or the like.

[0127] Fig. 4M is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4M is provided with a second display device 1680 on the far side (for example, the back panel of the housing 1190) as seen from the user 230 relative to the polarization separation member 101B of the space-floating image display device of Fig. 4G. The other configurations are the same as those of the space-floating image display device of Fig. 4G, so repeated explanations will be omitted.

[0128] In the configuration example shown in FIG. 4M, the second display device 1680 is provided behind the display position of the space-floating image 3, and its image display surface faces the space-floating image 3. With this configuration, from the user 230's perspective, the image of the second display device 1680 and the space-floating image 3, which are displayed at two different depth positions, can be viewed as overlapping images. In other words, the second display device 1680 can be said to be arranged so as to display an image in the direction of the user 230 who views the space-floating image 3. When the second display device 1680 is provided, it may be configured to be connected to other processing units such as the control unit 1110 as one component of the space-floating image display device 1000 of FIG. 3.

[0129] Note that the image light of the second display device 1680 of the space-floating image display device 1000 of FIG. 4M is viewed by the user 230 after passing through the polarization separator 101B. Therefore, in order for the image light of the second display device 1680 to more suitably pass through the polarization separator 101B, it is desirable that the image light output from the second display device 1680 be polarized in a vibration direction that the polarization separator 101B more suitably transmits. That is, it is desirable that the image light be polarized in the same vibration direction as the polarization of the image light output from the display device 1. For example, if the image light output from the display device 1 is S-polarized, it is desirable that the image light output from the second display device 1680 is also S-polarized. Furthermore, if the image light output from the display device 1 is P-polarized, it is desirable that the image light output from the second display device 1680 is also P-polarized.

[0130] The example of the space-floating image display device of FIG. 4M also has the same effect as the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L in that a second image is displayed behind the space-floating image 3. However, unlike the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L, in the example of the space-floating image display device of FIG. 4M, the luminous flux of image light for forming the space-floating image 3 does not pass through the second display device 1680. Therefore, the second display device 1680 does not need to be a transmissive self-luminous image display device, but may be a liquid crystal display, which is a two-dimensional flat display. The second display device 1680 may also be an organic EL display. Therefore, the example of the space-floating image display device of FIG. 4M can realize the space-floating image display device 1000 at a lower cost than the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L.

[0131] Here, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, a portion of the image light output from the display device 1 may be reflected by the polarization separation member 101B and travel toward the second display device 1680. This light (a portion of the image light) may be reflected again by the surface of the second display device 1680 and may be visually recognized by the user as stray light.

[0132] Therefore, to prevent this stray light, an absorptive polarizer may be provided on the surface of the second display device 1680. In this case, the absorptive polarizer may be an absorptive polarizer that transmits the polarized waves of the image light output from the second display device 1680 and absorbs polarized waves that are 90° out of phase with the polarized waves of the image light output from the second display device 1680. If the second display device 1680 is a liquid crystal display, an absorptive polarizer is also provided on the image output side of the liquid crystal display. However, if there is a cover glass (cover glass on the image display surface side) on the output surface of the absorptive polarizer on the image output side of the liquid crystal display, it is not possible to prevent stray light caused by reflection of the cover glass by light from outside the liquid crystal display. Therefore, it is necessary to separately provide the above-mentioned absorptive polarizer on the surface of the cover glass.

[0133] When an image is displayed on the second display device 1680, which is a two-dimensional flat display, the floating-in-space image 3 can be displayed as an image further in front of the image on the second display device 1680. In this case, the user 230 can simultaneously view two images at different depth positions. By displaying a character on the floating-in-space image 3 and a background on the second display device 1680, it is possible to provide the effect that the user 230 is viewing the space in which the character exists in three dimensions.

[0134] Furthermore, if the second display device 1680 displays both the background and an object such as a character, and then displays only the object such as the character moving to the front of the floating image 3, it is possible to provide the user 230 with a more effective surprise visual experience.

[0135] Next, Fig. 4N is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4N is a space-floating image display device that employs the optical system of Fig. 2D. In the space-floating image display device 1000 of Fig. 4N, similar to the examples of the space-floating image display device that employ the optical systems of Figs. 2A to 2C, image light that has passed through a transparent member 100 is formed in the air as a space-floating image 3. Furthermore, using sensing light from an aerial operation detection sensor 1351 that is arranged on the far side of the transparent member 100 as seen from the user, it is possible to detect operation of the space-floating image 3 by the user's finger 9004.

[0136] 2A to 2C, and in the example of the space-floating image display device employing the optical system of FIG. 2D, the space-floating image 3 is formed in front of the transparent member 100, and the operation of the space-floating image 3 by the user's finger can be detected using the sensing light of the mid-air operation detection sensor 1351 arranged on the back side of the transparent member 100 as seen from the user.

[0137] The space-floating image display device employing the optical system of Fig. 2D has a different optical system from the space-floating image display device in which the optical system of Fig. 2A to Fig. 2C is arranged on the back side of the transparent member 100 as seen from the user. However, the usability of the space-floating image display device employing the optical system of Fig. 2D as seen from the user is almost the same as that of the space-floating image display device employing the optical system of Fig. 2A to Fig. 2C.

[0138] Next, Fig. 4O is a diagram showing an example of the configuration of a space-floating image display device. Fig. 4O is a diagram showing the configuration of the internal optical system of the space-floating image display device 1000 of Fig. 4N. The space-floating image display device 1000 shown in Fig. 4O is equipped with an optical system corresponding to the optical system of Fig. 2D. The space-floating image display device 1000 shown in Fig. 4O is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward.

[0139] 4O, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. If the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230.

[0140] Here, the configuration of Fig. 4O will be compared with the configuration of Fig. 4A to confirm the differences. In Fig. 4A, the display device 1 and the space-floating image 3 are in a plane-symmetrical relationship with respect to the plane of the polarization separation member 101. In contrast, in Fig. 4O, the display device 1 and the space-floating image 3 are in a plane-symmetrical relationship with respect to the plane of the retroreflector 5. Furthermore, the configuration of Fig. 4A includes the retroreflector 2 and the λ / 4 plate 21, but these do not exist in Fig. 4O. Furthermore, while the presence of an absorbing polarizer 12 is more preferable in Fig. 4A, the absorbing polarizer 12 is not particularly necessary in Fig. 4O.

[0141] To replace the optical system of FIG. 2A in the configuration of FIG. 4A with the optical system of FIG. 2D and to replace it with the configuration of FIG. 4O, the following can be done. That is, the polarization separation member 101 in the configuration of FIG. 4A is replaced with the retroreflector 5, and the retroreflector 2 and the λ / 4 plate 21 are removed from the configuration of FIG. 4A. The absorptive polarizer 12 is optional. By performing a replacement based on this concept, the optical system of FIGS. 2A to 2C mounted in the configuration of the space-floating image display device of FIGS. 4A to 4G can be replaced with the optical system of FIG. 2D, and the space-floating image display device can be replaced with the optical system of FIG. 2D. In this case, the polarization separation member 101 in FIGS. 4A and 4B is replaced with the retroreflector 5, and the polarization separation member 101B in FIGS. 4C to 4G is replaced with the retroreflector 5.

[0142] For example, FIG. 4P is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4P is equipped with an optical system corresponding to the optical system shown in FIG. 2D. FIG. 4P shows the configuration of the space-floating image display device of FIG. 4B, with the optical system shown in FIG. 2A replaced with the optical system shown in FIG. 2D. The space-floating image display device 1000 shown in FIG. 4P is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (toward the user 230). That is, in FIG. 4P, the space-floating image display device is installed with the transparent member 100 facing the front of the device (toward the user 230). The space-floating image 3 is formed on the user 230 side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. If the mid-air operation detection sensor 1351 is installed as shown in the figure, it can detect the operation of the space-floating image 3 by the user 230's finger. 4P, the mid-air operation detection sensor 1351 can use the reflection of sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, the reflectivity of the nail is higher than that of the finger pad, so this configuration can improve the accuracy of touch detection.

[0143] According to the configuration of the space-floating image display device of FIGS. 4N to 4P, it is possible to realize a user-friendly space-floating image display device using the optical system of FIG. 2D.

[0144] <Display device> Next, the display device 1 of this embodiment will be described with reference to the drawings. The display device 1 of this embodiment includes a liquid crystal display panel 11 as an image display element 11, and a light source device 13 that constitutes a light source for the liquid crystal display panel 11. In Fig. 5, the light source device 13 is shown together with the liquid crystal display panel 11 as an exploded perspective view.

[0145] As shown by arrow 30 in Fig. 5, the image display element 11, which is a liquid crystal display panel, receives an illumination light beam from a light source device 13, which is a backlight device, that has narrow-angle diffusion characteristics, i.e., has strong directionality (in other words, linearity) and characteristics similar to laser light with a polarization plane aligned in one direction. The image display element 11, which is a liquid crystal display panel 11, modulates the received illumination light beam in accordance with an input video signal. The modulated image light is reflected by the retroreflector 2 and passes through a transparent member 100 to form a real image, a floating image (see Fig. 1).

[0146] 5, the display device 1 is configured with a light source device 13 and a liquid crystal display panel 11, a light redirection panel 54 that controls the directional characteristics of the light beam emitted from the light source device 13, and a narrow-angle diffuser (not shown) as needed. Specifically, polarizing plates are provided on both sides of the liquid crystal display panel 11, and as indicated by arrow 30 in FIG. 5, image light of a specific polarization is emitted with its intensity modulated by a video signal. This allows a desired image to be projected as highly directional (linear) light of a specific polarization via the light redirection panel 54 toward the retroreflector 2. After being reflected by the retroreflector 2, the light is transmitted toward the eyes of a monitor outside the store (space) shown in FIG. 1, forming the floating image 3. A protective cover 50 (see FIGS. 6 and 7) may be provided on the surface of the light redirection panel 54.

[0147] <Display device example 1> FIG. 6 shows an example of a specific configuration of the display device 1. In FIG. 6, a liquid crystal display panel 11 and a light direction conversion panel 54 are disposed on the light source device 13 shown in FIG. 5. This light source device 13 is configured by, for example, housing LED elements 201 and a light guide 203 inside a plastic case or the like. As shown in FIG. 5 and other figures, the end surface of the light guide 203 is provided with a lens shape whose cross-sectional area gradually increases toward the light receiving section in order to convert divergent light from each LED element 201 into a substantially parallel beam. The lens shape has an effect of gradually reducing the divergence angle by multiple total reflections during propagation inside. The liquid crystal display panel 11 constituting the display device 1 is attached to the top surface of the display device 1. In addition, an LED substrate 202 mounting the LED elements 201, which are semiconductor light sources, and their control circuits is attached to one side surface of the light source device 13 (the left end surface in this example). A heat sink, which is a member for cooling the heat generated by the LED elements 201 and the control circuit, may be attached to the outer surface of the LED substrate 202.

[0148] The liquid crystal display panel 11 is mounted on the top surface of the case of the light source device 13 through a frame (not shown) for the liquid crystal display panel 11. The frame is also fitted with an FPC (Flexible Printed Circuits) (not shown) electrically connected to the liquid crystal display panel 11. The liquid crystal display panel 11, which is the image display element 11, generates a display image by modulating the intensity of transmitted light in conjunction with the LED elements 201, which are solid-state light sources, based on a control signal from a control circuit (image control unit 1160 in FIG. 3) constituting the electronic device. The generated image light has a narrow diffusion angle and contains only specific polarization components, resulting in a novel image display device similar to a surface-emitting laser image source driven by a video signal. Currently, it is technically and safety-wise impossible to obtain a laser beam of the same size as the image obtained by the display device 1 described above using a laser device. Therefore, in this embodiment, light similar to the surface-emitting laser image light described above is obtained from a beam of light from a general light source, such as an LED element.

[0149] Next, the configuration of the optical system housed in the case of light source device 13 will be described in detail with reference to Fig. 6 and Fig. 7. Because Fig. 6 and Fig. 7 are cross-sectional views, only one of the multiple LED elements 201 constituting the light source is shown, and this is converted into approximately parallel light (collimated light) by the shape of light-receiving end surface 203a of light guide 203. For this reason, the light-receiving portion of the light guide end surface and LED element 201 are attached while maintaining a predetermined positional relationship.

[0150] Each light guide 203 is formed of a translucent resin such as acrylic. The LED light receiving surface at the end of light guide 203 has a cone-shaped outer periphery obtained by rotating a parabolic cross section, and the top of the light guide 203 has a concave portion with a convex portion (i.e., a convex lens surface) formed in the center, and the center of the flat portion has a convex lens surface (or a concave lens surface) that protrudes outward (not shown). The outer shape of the light receiving portion of the light guide to which LED element 201 is attached is a parabolic shape that forms a cone-shaped outer periphery, and is set within an angle range that allows total reflection within the light that is emitted from the LED element toward the periphery, or a reflective surface is formed.

[0151] On the other hand, the LED elements 201 are arranged at predetermined positions on the surface of the circuit board, that is, the LED substrate 202. The LED substrate 202 is arranged and fixed so that the LED elements 201 on the surface are positioned in the center of the recessed portion described above with respect to the light-receiving end surface 203a, that is, the LED collimator.

[0152] According to this configuration, the shape of the light-receiving end surface 203a of the light guide 203 makes it possible to extract the light emitted from the LED element 201 as approximately parallel light, thereby improving the efficiency of use of the generated light.

[0153] As described above, the light source device 13 is configured by attaching a light source unit having a plurality of LED elements 201 arranged as light sources to the light-receiving end surface 203a, which is a light-receiving section provided on the end surface of the light guide 203, and the divergent light beams from the LED elements 201 are converted into approximately parallel light by the lens shape of the light-receiving end surface 203a of the light guide 203, which is guided inside the light guide 203 as shown by the arrow, and emitted by the light beam direction conversion means 204 towards the liquid crystal display panel 11, which is arranged approximately parallel to the light guide 203. By optimizing the distribution (in other words, density) of the light beam direction conversion means 204 depending on the shape of the inside or surface of the light guide 203, it is possible to control the uniformity of the light beam incident on the liquid crystal display panel 11.

[0154] The light beam direction conversion means 204 described above emits the light beam propagated inside the light guide 203 toward the liquid crystal display panel 11 disposed approximately parallel to the light guide 203, by using the shape of the surface of the light guide 203 or by providing a portion with a different refractive index inside the light guide 203. At this time, when the liquid crystal display panel 11 is faced directly at the center of the screen and the viewpoint is positioned at the same position as the diagonal dimension of the screen, if the relative brightness ratio between the center of the screen and the periphery of the screen is 20% or more, there is no practical problem, and if it exceeds 30%, it will be an even better characteristic.

[0155] 6 is a cross-sectional layout diagram illustrating the configuration and operation of a light source of this embodiment that performs polarization conversion in light source device 13 including light guide 203 and LED element 201 described above. In Fig. 6, light source device 13 is composed of light guide 203 formed of, for example, plastic or the like and having light beam direction conversion means 204 on its surface or inside, LED element 201 as a light source, reflective sheet 205, retardation plate 206, lenticular lens, etc. Attached to the top surface of light source device 13 is liquid crystal display panel 11 that has polarizing plates on the light source light entrance surface and the image light exit surface.

[0156] Furthermore, a film- or sheet-like reflective polarizing plate 49 is provided on the light source light incidence surface (the lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, and selectively reflects one polarized wave (e.g., P wave) 212 of the natural light beam 210 emitted from the LED element 201. The reflected light is reflected again by a reflective sheet 205 provided on one surface (the lower surface in the figure) of the light guide 203, and directed toward the liquid crystal display panel 11. Therefore, a retardation plate (λ / 4 plate) is provided between the reflective sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49. The reflected light (reflected light beam) is reflected by the reflective sheet 205 and passes through the retardation plate (λ / 4 plate) twice, thereby converting it from P polarized light to S polarized light. This improves the utilization efficiency of the light source light as image light. The image light beam, the light intensity of which has been modulated by the image signal in the liquid crystal display panel 11, is emitted as shown by the arrow 213 in Fig. 6 and enters the retroreflector 2. After reflection by the retroreflector 2, a real image, a floating image in space, can be obtained.

[0157] 7 is a cross-sectional layout diagram illustrating the configuration and operation of a light source of this embodiment that performs polarization conversion in light source device 13 including light guide 203 and LED elements 201, similar to Fig. 6. Light source device 13 is similarly composed of light guide 203 formed of, for example, plastic and having light beam direction conversion means 204 on its surface or inside, LED elements 201 as a light source, reflective sheet 205, retardation plate 206, lenticular lens, etc. Attached to the top surface of light source device 13 is liquid crystal display panel 11 that has polarizing plates on the light source light entrance surface and the image light exit surface.

[0158] A film- or sheet-like reflective polarizing plate 49 is provided on the light source light incidence surface (the lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, and selectively reflects one polarized wave (e.g., S wave) 211 of the natural light beam 210 emitted from the LED element 201. That is, in the example of FIG. 7, the selective reflection characteristics of the reflective polarizing plate 49 are different from those in FIG. 7. The reflected light is reflected by a reflective sheet 205 provided on one surface (the lower surface in the figure) of the light guide 203 and returns to the liquid crystal display panel 11. A retardation plate (λ / 4 plate) is provided between the reflective sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49. The reflected light (reflected light beam) is reflected by the reflective sheet 205 and passes through the retardation plate (λ / 4 plate) twice, converting it from S-polarized light to P-polarized light. This improves the utilization efficiency of the light source light as image light. The image light beam, which has been intensity-modulated by the image signal on the liquid crystal display panel 11, is emitted as shown by the arrow 214 in Fig. 7 and enters the retroreflector 2. After being reflected by the retroreflector 2, a real image, a floating image in space, can be obtained.

[0159] In the light source device 13 shown in Figures 6 and 7, in addition to the function of the polarizer provided on the light incident surface of the corresponding liquid crystal display panel 11, a reflective polarizer reflects the polarized light component on one side. Therefore, the theoretically obtainable contrast ratio is the reciprocal of the cross transmittance of the reflective polarizer multiplied by the reciprocal of the cross transmittance obtained by the two polarizers attached to the liquid crystal display panel 11. This results in high contrast performance. In fact, experiments have confirmed that the contrast performance of the displayed image is improved by more than 10 times. As a result, high-quality images comparable to those of self-luminous organic EL devices are obtained.

[0160] <Display device example 2> 8 shows another example of the specific configuration of the display device 1. The light source device 13 of this display device 1 is configured by housing LEDs, a collimator, a composite diffusion block, a light guide, etc. in a case made of, for example, plastic, and a liquid crystal display panel 11 is attached to the upper surface of the light source device 13. Also, LED elements 201, which are semiconductor light sources, and an LED board 202, on which a control circuit for the LED elements 201 is mounted, are attached to one side of the case of the light source device 13, and a heat sink 103, which is a member for cooling heat generated by the LED elements 201 and the control circuit, is attached to the outer surface of the LED board 202.

[0161] The liquid crystal display panel frame attached to the top surface of the case of the light source device 13 is configured to have attached thereto the liquid crystal display panel 11 attached to the frame, and further to have attached thereto an FPC 403 electrically connected to the liquid crystal display panel 11. That is, the liquid crystal display panel 11, which is the image display element 11, generates a display image together with the LED elements 201, which are solid-state light sources, by modulating the intensity of transmitted light based on a control signal from a control circuit (not shown) that constitutes the electronic device.

[0162] <Display device example 3> Next, another example of the specific configuration of the display device 1 (Example 3 of the display device) will be described with reference to Fig. 9. The light source device of this display device 1 converts a divergent beam of light (a mixture of P-polarized and S-polarized light) from the LED 201 into a substantially parallel beam by a collimator (LED collimator) 18, and reflects the parallel beam toward the liquid crystal display panel 11 by the reflecting surface of the reflective light guide 304. The reflected light is incident on a reflective polarizer 49 disposed between the liquid crystal display panel 11 and the reflective light guide 304. The reflective polarizer 49 transmits light of a specific polarization (e.g., P-polarized light) and causes the transmitted polarized light to be incident on the liquid crystal display panel 11. Here, light of polarization other than the specific polarization (e.g., S-polarized light) is reflected by the reflective polarizer 49 and directed toward the reflective light guide 304 again.

[0163] The reflective polarizing plate 49 is installed at an angle with respect to the liquid crystal display panel 11 so that the reflective polarizing plate 49 is not perpendicular to the chief ray of light from the reflective surface of the reflective light guide 304. The chief ray of the light reflected by the reflective polarizing plate 49 is incident on the transmission surface of the reflective light guide 304. The light that has entered the transmission surface of the reflective light guide 304 passes through the back surface of the reflective light guide 304, passes through the λ / 4 plate 270 which is a retardation plate, and is reflected by the reflector 271. The light reflected by the reflector 271 passes through the λ / 4 plate 270 again, and passes through the transmission surface of the reflective light guide 304. The light that has passed through the transmission surface of the reflective light guide 304 is incident on the reflective polarizing plate 49 again.

[0164] At this time, the light that re-enters the reflective polarizer 49 has passed through the λ / 4 plate 270 twice, and therefore its polarization has been converted to a polarization (for example, P-polarized light) that is transmitted through the reflective polarizer 49. Therefore, the light whose polarization has been converted passes through the reflective polarizer 49 and enters the liquid crystal display panel 11. Note that with regard to the polarization design related to the polarization conversion, the polarization may be configured in reverse from the above explanation (S-polarized light and P-polarized light may be reversed).

[0165] As a result, the light from the LED 201 is aligned to a specific polarization (for example, P polarization), enters the liquid crystal display panel 11, and is brightness-modulated in accordance with the video signal to display an image on the panel surface. As in the above example, there are multiple LEDs 201 that constitute the light source, and these are attached to predetermined positions relative to the corresponding collimators 18 of the multiple collimators 18. However, since Figure 9 is a vertical cross section, only one LED 201 and one collimator 18 are shown.

[0166] Each of the collimators 18 is formed of, for example, a translucent resin such as acrylic or glass. The collimator 18 may have a cone-shaped outer peripheral surface obtained by rotating a parabolic cross section. The collimator 18 may have a concave portion with a convex portion (i.e., a convex lens surface) formed in the center of the apex (the side facing the LED substrate 202) of the collimator 18. The collimator 18 may have a convex lens surface protruding outward (or a concave lens surface recessed inward) in the center of the flat portion (the side opposite the apex). The parabolic surface forming the cone-shaped outer peripheral surface of the collimator 18 is set within an angle range that allows the light emitted from the LED 201 in the peripheral direction to be totally reflected therein, or a reflective surface is formed therein.

[0167] The LEDs 201 are arranged at predetermined positions on the surface of the LED substrate 202, which is the circuit board. The LED substrate 202 is arranged and fixed to the collimator 18 so that the LEDs 201 on the surface are positioned at the center of the apex of the conical convex shape (or in the concave portion if the apex has a concave portion).

[0168] With this configuration, the collimator 18 focuses the light emitted from the LED 201, particularly the light emitted from the central portion, into parallel light by the convex lens surface that forms the outer shape of the collimator 18. Light emitted from other portions toward the periphery is reflected by the parabolic surface that forms the outer peripheral surface of the cone shape of the collimator 18, and is similarly focused into parallel light. In other words, the collimator 18, which has a convex lens in its center and a parabolic surface formed on its periphery, makes it possible to extract almost all of the light generated by the LED 201 as parallel light, thereby improving the utilization efficiency of the generated light.

[0169] Furthermore, the light converted into approximately parallel light by the collimator 18 shown in FIG. 9 is reflected by the reflective light guide 304. Due to the action of the reflective polarizer 49, light of a specific polarization of the light is transmitted through the reflective polarizer 49, while light of the other polarization reflected by the reflective polarizer 49 is transmitted again through the light guide 304. The light is reflected by the reflector 271, which is located opposite the liquid crystal display panel 11 with respect to the reflective light guide 304. At this time, the light is polarized and converted twice by passing through the λ / 4 plate 270, which is a retardation plate. The light reflected by the reflector 271 is transmitted again through the light guide 304 and enters the reflective polarizer 49 provided on the opposite surface. Since the incident light has been polarization-converted, it is transmitted through the reflective polarizer 49 and enters the liquid crystal display panel 11 with its polarization direction aligned. As a result, all of the light from the light source can be utilized, thereby doubling the geometrical optical utilization efficiency of light. Furthermore, since the degree of polarization (extinction ratio) of the reflective polarizer 49 is also included in the extinction ratio of the entire system, the use of the light source device of this embodiment significantly improves the contrast ratio of the entire display device. Note that the reflection diffusion angle of light at each reflective surface can be adjusted by adjusting the surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflector 271. The surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflector 271 can be adjusted for each design to optimize the uniformity of the light incident on the liquid crystal display panel 11.

[0170] It should be noted that the λ / 4 plate 270, which is the retardation plate in Fig. 9, does not necessarily have to have a phase difference of λ / 4 with respect to polarized light that is perpendicularly incident on the λ / 4 plate 270. In the configuration of Fig. 9, any retardation plate may be used as long as the phase changes by 90° (λ / 2) when polarized light passes through it twice. The thickness of the retardation plate may be adjusted according to the incident angle distribution of the polarized light.

[0171] <Display device example 4> Furthermore, another example (display device example 4) of the configuration of the optical system such as the light source device of display device 1 will be described with reference to FIG. 10 . Display device example 4 is a configuration example in which a diffusion sheet is used instead of reflective light guide 304 in the light source device of display device example 3. Specifically, two optical sheets (in other words, diffusion sheets) that convert the diffusion characteristics in the vertical and horizontal directions of the drawing (front and back directions in the drawing, not shown) are used on the light emission side of collimator 18. The two optical sheets are shown as optical sheet 207A and optical sheet 207B. Light from collimator 18 is made to enter between the two optical sheets.

[0172] The optical sheet may be a single sheet instead of a two-sheet configuration. In the case of a single sheet configuration, the vertical and horizontal diffusion characteristics are adjusted by the fine shapes on the front and back surfaces of the single optical sheet. Alternatively, multiple diffusion sheets may be used to share the functions. In the example of FIG. 10 , the reflection and diffusion characteristics due to the front and back shapes of optical sheets 207A and 207B can be optimally designed using the number of LEDs 201, the divergence angle from LED substrate 202, and the optical specifications of collimator 18 as design parameters so that the surface density of the light beam emitted from liquid crystal display panel 11 is uniform. In other words, in the example of FIG. 10 , the diffusion characteristics are adjusted by the surface shapes of multiple diffusion sheets instead of light guides.

[0173] In the example of FIG. 10, polarization conversion is performed in the same manner as in the display device example 3 described above. That is, in the example of FIG. 10, reflective polarizing plate 49 may be configured to have the property of reflecting S-polarized light (transmitting P-polarized light). In this case, the reflective polarizing plate 49 transmits P-polarized light out of the light emitted from LED 201, which is the light source, and the transmitted light enters liquid crystal display panel 11. The reflective polarizing plate 49 reflects S-polarized light out of the light emitted from LED 201, which is the light source, and the reflected light passes through retardation plate 270 shown in FIG. 10. The light that passes through retardation plate 270 is reflected by reflector 271. The light reflected by reflector 271 passes through retardation plate 270 again and is converted to P-polarized light. The polarization-converted light passes through reflective polarizing plate 49 and enters liquid crystal display panel 11.

[0174] It should be noted that the λ / 4 plate 270, which is the retarder in FIG. 10, does not necessarily have to have a phase difference of λ / 4 with respect to polarized light that is perpendicularly incident on the λ / 4 plate 270. In the configuration of FIG. 10, any retarder that changes the phase by 90° (λ / 2) when polarized light passes through it twice will suffice. The thickness of the retarder may be adjusted according to the distribution of incident angles of the polarized light. It should be noted that in FIG. 10 as well, the polarization design for polarization conversion may be configured in reverse (reversing the S-polarized light and P-polarized light) from the above explanation.

[0175] In a typical TV device, the light emitted from the LCD panel 11 has similar diffusion characteristics in both the horizontal direction of the screen (shown by the X-axis in FIG. 12(a)) and the vertical direction of the screen (shown by the Y-axis in FIG. 12(b)). In contrast, the diffusion characteristics of the light beam emitted from the LCD panel 11 of this embodiment are 1 / 5 of the 62-degree viewing angle of a typical TV device, as shown in Example 1 of FIG. 12, when the viewing angle at which the luminance is 50% of that at a front view (angle of 0 degrees) is set to 13 degrees. Similarly, the vertical viewing angle is asymmetric between the top and bottom, and the reflection angle and the area of ​​the reflective surface of the reflective light guide are optimized to keep the upper viewing angle to about 1 / 3 of the lower viewing angle. As a result, the amount of image light directed toward the monitoring direction is significantly improved compared to conventional LCD TVs, with brightness being more than 50 times higher.

[0176] Furthermore, assuming the viewing angle characteristics shown in Example 2 in Figure 12, if the viewing angle at which brightness is 50% of that when viewed from the front (angle of 0 degrees) is set to 5 degrees, this will be 1 / 12 of the 62 degrees of devices used for general TV applications. Similarly, the vertical viewing angle is optimized by optimizing the reflection angle and the area of ​​the reflective surface of the reflective light guide so that the viewing angle is approximately 1 / 12 of that of devices used for general TV applications, with equal viewing angle both above and below. As a result, the amount of image light directed in the monitoring direction is significantly improved compared to conventional LCD TVs, and brightness is more than 100 times greater.

[0177] As described above, by setting the viewing angle to a narrow angle, the amount of luminous flux directed in the monitoring direction can be concentrated, significantly improving the efficiency of light utilization. As a result, even when using a liquid crystal display panel for general TV applications, by controlling the light diffusion characteristics of the light source device, it is possible to achieve a significant improvement in brightness with similar power consumption, making it possible to create a video display device that is compatible with information display systems facing bright outdoor environments.

[0178] When using a large LCD display panel, the overall brightness of the screen can be improved by directing the light from the periphery of the screen inward so that it is directed toward the observer when the observer is facing the center of the screen. Figure 11 shows the convergence angle between the long and short sides of the panel when the observer's distance from the panel, L, and the panel size (screen ratio 16:10) are used as parameters. When monitoring with the screen in portrait orientation, the convergence angle can be set to match the short side. For example, when using a 22-inch panel in portrait orientation and the monitoring distance is 0.8 m, a convergence angle of 10 degrees will allow the image light from the four corners of the screen to be effectively directed toward the observer.

[0179] Similarly, when monitoring with a 15-inch panel in portrait orientation, if the monitoring distance is 0.8 m, a convergence angle of 7 degrees will allow the image light from the four corners of the screen to be effectively directed towards the monitor. As mentioned above, depending on the size of the LCD panel and whether it is used portrait or landscape, the overall brightness of the screen can be improved by directing the image light from the periphery of the screen towards the monitor who is in the optimum position to monitor the centre of the screen.

[0180] As shown in Figure 9, the basic configuration involves a light source device directing a light beam with a narrow angle of directionality to a liquid crystal display panel 11, which is then luminance-modulated according to a video signal. The video information displayed on the screen of the liquid crystal display panel 11 is then reflected by a retroreflector, and the resulting floating image is displayed indoors or outdoors via a transparent member 100.

[0181] By using the display device and light source device according to the embodiment of the present invention described above, it is possible to realize a space floating image display device with higher light utilization efficiency.

[0182] <Example of image display processing in a space floating image display device> Next, an example of a problem solved by the image processing of this embodiment will be described with reference to Fig. 13A. In the space-floating image display device 1000 (Fig. 3, Fig. 4A to Fig. 4P), the far side of the space-floating image 3 from the user's perspective is inside the housing of the space-floating image display device 1000, and if it is sufficiently dark, the user will visually recognize that the background of the space-floating image 3 is black.

[0183] Here, an example of displaying a character "panda" 1525 in the space floating image 3 will be described using Fig. 13A. First, the image control unit 1160 in Fig. 3 distinguishes and recognizes the pixel area where the image of the character "panda" 1525 is drawn from the transparent information area 1520 which is the background image, for an image including the pixel area where the image of the character "panda" 1525 is drawn and the transparent information area 1520 which is the background image, as shown in Fig. 13A(1).

[0184] A method for distinguishing and recognizing the character image from the background image is, for example, to configure the image processing of the video control unit 1160 so that the background image layer and the layer of the character image in front of the background image layer can be processed as separate layers, and the character image and background image can be distinguished and recognized based on the superimposition relationship when these layers are combined.

[0185] Here, the image control unit 1160 recognizes black pixels that depict objects such as character images and transparent information pixels as different information. However, it is assumed that both the black pixels that depict objects and the transparent information pixels have a luminance of 0. In this case, when the space floating image 3 is displayed, there is no difference in luminance between the pixels that depict black in the image of the character "panda" 1525 and the pixels of the transparent information region 1520, which is the background image. Therefore, in the space floating image 3, as shown in FIG. 13A(2), neither the pixels that depict black in the image of the character "panda" 1525 nor the pixels of the transparent information region 1520 have luminance, and they are visually recognized by the user as the same optically black space. In other words, the black portions of the image of the character "panda" 1525, which is an object, blend into the background, and only the non-black portions of the character "panda" 1525 are recognized as floating in the display region of the space floating image 3.

[0186] An example of image processing according to this embodiment will be described with reference to FIG. 13B. FIG. 13B is a diagram illustrating an example of image processing that more suitably resolves the issue of the black image region of the object blending into the background, as described in FIG. 13A. In FIGS. 13B(1) and 13B(2), the upper side shows the display state of the floating image 3 in space, and the lower side shows the input / output characteristics of the image processing of the image of the object. Note that the image of the object (character "panda" 1525) and the corresponding data may be read from the storage unit 1170 or memory 1109 in FIG. 3, or may be input from the video signal input unit 1131, or may be acquired via the communication unit 1132.

[0187] In the state shown in Figure 13B(1), the input / output characteristics of the image processing of the object image are in a linear state with no particular adjustment. In this case, the display state is the same as that shown in Figure 13A(2), and the black image area of ​​the object blends into the background. In contrast, in Figure 13B(2), the video control unit 1160 of this embodiment adjusts the input / output characteristics of the image processing of the image of the object (character "panda" 1525) to the input / output characteristics shown in the lower part.

[0188] That is, the video control unit 1160 performs image processing with input / output characteristics on the image of the object (character "panda" 1525), which has a characteristic of converting pixels in low-brightness areas of the input image into output pixels with increased brightness values. After the image of the object (character "panda" 1525) has been subjected to image processing with the input / output characteristics, a video including the image of the object (character "panda" 1525) is input to the display device 1 and displayed. Then, as shown in the upper part of FIG. 13B(2), the display state of the floating in space image 3 is such that the brightness of pixel areas depicting black in the image of character "panda" 1525 increases. This allows the user to distinguish the areas depicting black among the areas depicting the image of character "panda" 1525 without them blending into the black background, making it possible to display the object more appropriately.

[0189] 13B(2), the area displaying the image of the character "panda" 1525, which is an object, can be distinguished from the black background inside the housing of the space-floating image display device 1000 through the window, improving the visibility of the object. Therefore, for example, even if the object includes pixels with a brightness value of 0 before the image processing (i.e., when the image of the object or the corresponding data is read from the storage unit 1170 or memory 1109 in FIG. 3, or when the image of the object is input from the video signal input unit 1131, or when the data of the object is obtained via the communication unit 1132, etc.), the image processing of the input / output characteristics by the video control unit 1160 converts the object into an object with a brightness value of 0 increased for the pixels in the low-brightness area, and then the object is displayed on the display device 1 and converted into a space-floating image 3 by the optical system of the space-floating image display device 1000.

[0190] That is, the pixels that make up the object after image processing of the input / output characteristics are converted to a state in which they do not include pixels with a brightness value of 0, and then they are displayed on the display device 1 and converted into a floating image 3 in space by the optical system of the floating image display device 1000.

[0191] In the image processing of Figure 13B(2), a method for applying image processing with the input / output characteristics of Figure 13B(2) only to the image area of ​​the object (character "panda" 1525) is, for example, to configure the image processing of the video control unit 1160 so that the background image layer and the layer of the character image in front of the background image layer can be processed as separate layers, and the image processing with the input / output characteristics of Figure 13B(2) is applied to the character image layer, while not applying this image processing to the background image layer.

[0192] Then, by combining these layers, image processing with a characteristic of raising the low-brightness areas of the input image is performed only on the character image, as shown in Fig. 13B(2). Alternatively, after the character image layer and background image layer are combined, image processing with the input / output characteristics of Fig. 13B(2) may be performed only on the character image area.

[0193] Furthermore, the input / output image characteristics used in the image processing for boosting low-luminance regions of the input / output characteristics for the input image are not limited to the example shown in FIG. 13B(2). Any image processing for boosting low luminance may be used, including so-called brightness adjustment. Alternatively, image processing for improving visibility by controlling the gain that changes the weighting of Retinex processing, as disclosed in International Publication WO 2014 / 162533, may be performed.

[0194] According to the image processing of FIG. 13B(2) described above, it is possible to make the user aware of areas where black is drawn among areas where images of characters, objects, etc. are drawn without blending into the black background, thereby realizing a more suitable display.

[0195] 13A and 13B, the problems and more suitable image processing were explained using the space-floating image display device in which the background appears black (for example, the space-floating image display device 1000 in FIGS. 4A to 4G, or the space-floating image display device 1000 in the state where the rear window is shielded in FIGS. 4I and 4J). However, the image processing is also effective for devices other than these space-floating image display devices.

[0196] Specifically, in the space-floating image display device 1000 of Fig. 4H, or in Fig. 4I and Fig. 4J where the rear window is not shaded, the background of the space-floating image 3 is not black, but the scenery behind the space-floating image display device 1000 through the window. In this case, the same problems as those described in Fig. 13A and Fig. 13B exist.

[0197] That is, the part of the image of the character "panda" 1525, which is an object, that is drawn in black will blend into the scenery behind the space-floating image display device 1000 through the window. In this case too, by using the image processing of Fig. 13B(2), the part of the image of the character "panda" 1525, which is an object, that is drawn in black will be recognized as being distinct from the scenery behind the space-floating image display device 1000 through the window, improving the visibility of the object.

[0198] That is, by using the image processing of FIG. 13B(2), the area displaying the image of the object character "panda" 1525 can be recognized as distinct from the scenery behind the space floating image display device 1000 through the window, and it becomes possible to more easily recognize that the object character "panda" 1525 is in front of the scenery, improving the visibility of the object.

[0199] 4K, 4L, and 4M, as described above, when another image (such as an image from the transmissive self-luminous image display device 1650 or an image from the second display device 1680) is displayed at a position different in depth from the space-floating image 3, the background of the space-floating image 3 is not black, but the other image. In this case, the problems explained in FIGS. 13A and 13B also exist.

[0200] That is, the part of the image of the character "panda" 1525, which is an object, that is drawn in black will blend into the other image that is displayed at a different depth from the floating image in space 3. In this case as well, by using the image processing of Fig. 13B(2), the part of the image of the character "panda" 1525, which is an object, that is drawn in black will be able to be recognized as distinct from the other image, improving the visibility of the object.

[0201] In other words, by using the image processing of Figure 13B (2), the area displaying the image of the object character "panda" 1525 can be recognized as distinct from the other image, and it can be more easily recognized that the object character "panda" 1525 is in front of the other image, improving the visibility of the object.

[0202] An example of the image display process of this embodiment will be described with reference to Fig. 13C. Fig. 13C shows an example of the image display of this embodiment in which the space floating image 3 and a second image 2050, which is another image, are simultaneously displayed. The second image 2050 may correspond to the image displayed by the transmissive self-luminous image display device 1650 of Fig. 4K or Fig. 4L. The second image 2050 may also correspond to the image displayed by the second display device 1680 of Fig. 4M.

[0203] That is, the example of the image display in Fig. 13C shows a specific example of the image display examples of the space-floating image display device 1000 in Figs. 4K, 4L, and 4M. In the example of this figure, a bear character is displayed in the space-floating image 3. The area other than the bear character in the space-floating image 3 is displayed in black, and becomes transparent as a space-floating image. In addition, the second image 2050 is a background image in which a plain, a mountain, and a sun are drawn.

[0204] 13C, the floating in space image 3 and the second image 2050 are displayed at different depth positions. When the user 230 views the two images, the floating in space image 3 and the second image 2050, in the line of sight of the arrow 2040, the user 230 can view the two images in a superimposed state. Specifically, the bear character of the floating in space image 3 appears to be superimposed in front of the background of plains, mountains, and the sun depicted in the second image 2050.

[0205] Here, since the space floating image 3 is formed as a real image in the air, when the user 230 moves his / her viewpoint slightly, he / she can recognize the depth of the space floating image 3 and the second image 2050 due to parallax. Therefore, the user 230 can get a stronger sense of floating in space from the space floating image 3 while viewing the two images in an overlapping state.

[0206] An example of the video display process of this embodiment will be described with reference to Fig. 13D. Fig. 13D(1) is a diagram of the floating in space image 3, from the example of the video display of this embodiment in Fig. 13C, as seen from the line of sight of the user 230. Here, a bear character is displayed in the floating in space image 3. The area other than the bear character in the floating in space image 3 is displayed in black, and becomes transparent as a floating in space image.

[0207] 13D(2) is a diagram showing the second image 2050 in the example of the video display of this embodiment in FIG. 13C as viewed from the line of sight of the user 230. In the example of this figure, the second image 2050 is a background image in which a plain, a mountain, and a sun are drawn.

[0208] 13D(3) is a diagram showing the state in which the second image 2050 and the floating in space image 3 appear superimposed in the line of sight of the user 230, among the example of image display of this embodiment in Fig. 13C. Specifically, the bear character of the floating in space image 3 appears superimposed in front of the background of plains, mountains, and the sun drawn in the second image 2050.

[0209] Here, when simultaneously displaying the space-floating image 3 and the second image 2050, it is desirable to pay attention to the balance of brightness between the two images in order to ensure better visibility of the space-floating image 3. If the second image 2050 is too bright compared to the brightness of the space-floating image 3, the displayed image of the space-floating image 3 will be transparent, and the second image 2050, which is the background, will be strongly visible through it.

[0210] Therefore, the output of the light source of the spatially floating image 3 and the display image brightness of the display device 1, and the output of the light source of the display device displaying the second image 2050 and the display image brightness of the display device should be set so that at least the brightness per unit area of ​​the spatially floating image 3 at the display position of the spatially floating image 3 is greater than the brightness per unit area of ​​the image light that reaches the display position of the spatially floating image 3 from the second image 2050.

[0211] Note that this condition only needs to be satisfied when the space-floating image 3 and the second image 2050 are displayed simultaneously, and therefore when switching from the first display mode in which the space-floating image 3 is not displayed and only the second image 2050 is displayed to the second display mode in which the space-floating image 3 and the second image 2050 are displayed simultaneously, control may be performed to reduce the brightness of the second image 2050 by lowering the output of the light source of the display device that displays the second image 2050 and / or the display image brightness of the display device. These controls may be realized by the control unit 1110 in Fig. 3 controlling the display device 1 and the display device that displays the second image 2050 (the transmissive self-luminous image display device 1650 in Fig. 4K or Fig. 4L or the second display device 1680 in Fig. 4M).

[0212] Note that when switching from the above-described first display mode to the above-described second display mode, if control is performed to reduce the brightness of second image 2050, the brightness may be reduced uniformly across the entire screen of second image 2050. Alternatively, instead of reducing the brightness uniformly across the entire screen of second image 2050, the brightness reduction effect may be greatest in the portion where an object is displayed in space-floating image 3, and the brightness reduction effect may be gradually weakened around that portion. In other words, if the brightness of second image 2050 is reduced only in the portion where space-floating image 3 is visually recognized as being superimposed on second image 2050, the visibility of space-floating image 3 is sufficiently ensured.

[0213] Here, since the space floating image 3 and the second image 2050 are displayed at positions with different depths, when the user 230 slightly changes his / her viewpoint, the parallax causes a change in the superimposed position of the space floating image 3 relative to the second image 2050. Therefore, when switching from the above-mentioned first display mode to the above-mentioned second display mode, if the brightness is to be reduced unevenly across the entire screen of the second image 2050, it is not desirable to reduce the brightness sharply based on the outline of the object displayed in the space floating image 3, but rather it is desirable to perform a gradation process of the brightness reduction effect, which changes the brightness reduction effect stepwise depending on the position as described above.

[0214] In addition, in the space floating image display device 1000 where the position of the object displayed in the space floating image 3 is approximately at the center of the space floating image 3, the position where the brightness reduction effect of the gradation processing of the brightness reduction effect is greatest can be the center position of the space floating image 3.

[0215] According to the image display process of this embodiment described above, the user 230 can visually recognize the space floating image 3 and the second image 2050 more favorably.

[0216] Note that when displaying the space-floating image 3, control may be performed so as not to display the second image 2050. Since the visibility of the space-floating image 3 is improved when the second image 2050 is not displayed, this control is suitable for the space-floating image display device 1000 and the like, which are used in applications where the user must be able to reliably view the space-floating image 3 when the space-floating image 3 is displayed.

[0217] <Example 2> As Example 2 of the present invention, an example of another configuration example of the space-floating image display device will be described. Note that the space-floating image display device according to this example is obtained by changing the optical system stored in the space-floating image display device described in Example 1 to the optical system shown in FIG. 14(1) or FIG. 14(2). In this example, differences from Example 1 will be described, and repeated explanations of the same configuration as Example 1 will be omitted. Note that in the following description of this example, the predetermined polarized light and the other polarized light are polarized waves whose phases differ by 90° from each other.

[0218] Fig. 14(1) shows an example of an optical system and an optical path according to this embodiment. The optical system shown in Fig. 14(1) is configured such that the display device 1 is closer to the polarization separation member 101B in the optical system of Fig. 2C, making the entire optical system more compact. In Fig. 14(1), the components denoted by the same reference numerals as in Fig. 2C will not be described in detail again.

[0219] 14(1), similar to FIG. 2C, image light of a predetermined polarized light (P polarized light in the figure) emitted from display device 1 travels in a direction perpendicular to the image display surface of display device 1. Here, similar to FIG. 2C, polarization separation member 101B selectively transmits the predetermined polarized light (P polarized light in the figure) emitted from display device 1 and reflects the other polarized light (S polarized light in the figure).

[0220] Therefore, image light of a predetermined polarization (P-polarized light in the figure) traveling vertically from the image display surface of display device 1 passes through polarization separator 101B and reaches retroreflector 2 to which λ / 4 plate 21 is attached. The image light that is retroreflected by retroreflector 2 and travels again toward polarization separator 101B has passed through λ / 4 plate 21 twice, and is converted from the predetermined polarization (P-polarized light in the figure) at the time of emission from display device 1 to the other polarization (S-polarized light in the figure). The image light that travels again toward polarization separator 101B is the other polarization (S-polarized light in the figure), and is therefore reflected by polarization separator 101B toward the position where the user should be. The traveling direction of the image reflected by polarization separator 101B is determined based on the angle at which polarization separator 101B is disposed.

[0221] In the example of Figure 14(1), the image light traveling toward the polarization separation member 101B is reflected at a right angle by the polarization separation member 101B and travels as shown in the figure. The image light reflected by the polarization separation member 101B forms a space-floating image 3A. The space-floating image 3A can be viewed by the user from the direction of arrow A.

[0222] Here, due to the characteristics of retroreflection by the retroreflector 2, the optical path length of the image light emitted from the display device 1 until it reaches the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 until it reaches the formation position of the space floating image 3A. This relationship determines the formation position of the space floating image 3A in the traveling direction of the image light reflected by the polarization separation member 101B.

[0223] In the example of FIG. 14(1), the display device 1, the polarization separating member 101B, and the retroreflector 2 are arranged closer than in the example of FIG. 2C. This allows the entire optical system to be configured more compactly. However, the amount by which the space-floating image 3A protrudes from the optical system of FIG. 14(1) is not very large. For example, as an index of the amount by which the space-floating image 3A protrudes from the optical system, the figure shows the distance (L1 in the example of FIG. 14(1)) from the position where the central light beam of the image light is reflected by the polarization separating member 101B to the position where the image light forms the space-floating image 3A.

[0224] 14(1), the characteristics of P polarization and S polarization may be interchanged. Specifically, the predetermined polarization of the image light emitted from the display device 1 may be S polarization, and the characteristics of P polarization and S polarization may be interchanged in the reflection characteristics of the polarization separation member 101B. In this case, the P polarization and S polarization shown in the figure are both reversed, but the optical design, such as the optical path, can be realized in exactly the same way.

[0225] Next, Fig. 14(2) shows another example of an optical system and an optical path according to this embodiment. The optical system of Fig. 14(2) is a modified version of the optical system of Fig. 14(1) in order to increase the amount of the floating image projecting from the optical system while realizing the same compactness as the optical system of Fig. 14(1). In Fig. 14(2), the components with the same reference numerals as those in Fig. 14(1) will not be described in detail again.

[0226] 14(2), similar to FIG. 14(1), image light of a predetermined polarized light (P polarized light in the figure) emitted from the display device 1 travels in a direction perpendicular to the image display surface of the display device 1. Here, the polarization characteristics of the polarization separation member 101B are arranged 90 degrees differently from those in FIG. 14(1). The image light of a predetermined polarized light (P polarized light in the figure) traveling in a direction perpendicular to the image display surface of the display device 1 passes through the polarization separation member 101B.

[0227] 14(1), the image light passing through the polarization separation member 101B is not faced with the retroreflector 2 having the λ / 4 plate 21 attached thereto, but with the specular reflector 4 having the λ / 4 plate 21B attached thereto. Here, the reflection at the specular reflector 4 is specular reflection (also called regular reflection), not retroreflection.

[0228] Therefore, the image light that has passed through polarization separation member 101B is specularly reflected by specular reflector 4 to which λ / 4 plate 21B is attached. The image light that has been specularly reflected by specular reflector 4 and travels again toward polarization separation member 101B has been converted from the predetermined polarization (P polarization in the figure) at the time of emission from display device 1 to the other polarization (S polarization in the figure) by having passed through λ / 4 plate 21 twice. The image light that has traveled again toward polarization separation member 101B is the other polarization (S polarization in the figure), and is therefore reflected by polarization separation member 101B.

[0229] Here, because the orientation of the polarization separator 101B in Figure 14(2) is different from that in Figure 14(1), the image light reflected by the polarization separator 101B travels in the opposite direction from where the user should be. A retroreflector 2 with a λ / 4 plate 21C attached is disposed at the destination of the image light reflected by the polarization separator 101B. The image light is retroreflected by the retroreflector 2. The image light that is retroreflected by the retroreflector 2 and travels again toward the polarization separator 101B has been converted from the other polarized light (S-polarized light in the figure) to the specified polarized light (P-polarized light in the figure) by passing through the λ / 4 plate 21C twice.

[0230] The image light that travels back toward the polarization separation member 101B is of a predetermined polarization (P polarization in the figure), so it passes through the polarization separation member 101B and continues toward the location where the user should be. The image light that has passed through the polarization separation member 101B forms a space-floating image 3B. The space-floating image 3B can be easily viewed by the user from the direction of arrow A.

[0231] 14(2), similarly to FIG. 14(1), due to the characteristics of retroreflection by the retroreflector 2, the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 to reach the formation position of the space-floating image 3B. This relationship determines the formation position of the space-floating image 3B in the traveling direction of the image light transmitted through the polarization separation member 101B.

[0232] 14(2), the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 is longer than the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 in FIG. 14(1). This is because in the optical system of FIG. 14(2), an optical path that goes back and forth between the polarization separation member 101B and the specular reflector 4, which does not exist in the optical system of FIG. 14(1), is added to the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2.

[0233] As a result, in the optical system of Figure 14(2), the distance from the position where the central ray of the image light passes through the polarization separation member 101B to the position where the image light forms the space-floating image 3B (L2 in the example of Figure 14(2)) is significantly longer than the distance from the position where the central ray of the image light is reflected by the polarization separation member 101B to the position where the image light forms the space-floating image 3A (L1 in the example of Figure 14(1)) in the optical system of Figure 14(1).

[0234] 14(2), the characteristics of P-polarized light and S-polarized light may be interchanged. Specifically, the predetermined polarization of the image light emitted from the display device 1 may be S-polarized light, and the characteristics of P-polarized light and S-polarized light may be interchanged with respect to the reflection characteristics of the polarization separation member 101B. In this case, the P-polarized light and S-polarized light shown in the figure are both reversed, but the optical design, such as the optical path, can be realized in exactly the same way.

[0235] As described above, according to the optical systems of Fig. 14(1) and Fig. 14(2) in the second embodiment of the present invention, a more compact optical system can be realized. In particular, according to the optical system of Fig. 14(2), it is possible to increase the amount of the floating image projecting from the optical system, despite the more compact optical system.

[0236] When the optical system of Fig. 14(1) or Fig. 14(2) is incorporated into a space-floating image display device, it can be realized by replacing the optical system in the space-floating image display device described in Example 1 with the optical system of Fig. 14(1) or Fig. 14(2). Specifically, the optical system of Fig. 14(1) may be replaced with the optical system of the space-floating image display device of Fig. 4E, Fig. 4F, Fig. 4G, Fig. 4H, Fig. 4I, Fig. 4J, Fig. 4K, Fig. 4L, or Fig. 4M. In this case, since the optical system becomes compact, it is possible to make the housing of the space-floating image display device of each figure smaller.

[0237] Specifically, the optical system of Fig. 14(2) may be replaced with the optical system of the space-floating image display device of Fig. 4E, Fig. 4F, Fig. 4G, Fig. 4K, or Fig. 4L. In this case, it becomes possible to increase the amount of space-floating images projecting from the optical system. Also, since the optical system becomes compact, it is possible to make the housing of the space-floating image display device of each figure smaller.

[0238] Example 3 As the third embodiment of the present invention, a configuration example of a space-floating image display device will be described. The space-floating image display device according to this embodiment can similarly apply the configurations of the respective drawings explained in the first and second embodiments as a basic configuration. In this embodiment, differences from the first and second embodiments will be mainly explained, and repeated explanations of the same configurations as the first and second embodiments will be omitted.

[0239] [Comparative example: Space-floating image display device inside a vehicle] FIG. 15 shows a configuration example of a space-floating image display device installed in a vehicle as a comparative example for Example 3. In this comparative example, a space-floating image display device 1000 capable of manipulating aerial images in mid-air is installed in a vehicle (in other words, a vehicle) such as an automobile. A seat 900 is installed in a vehicle body 902 of the vehicle. FIG. 15 shows a schematic diagram of the vehicle seen from the side. In FIG. 15, a user 230, who is a passenger, is seated in the vehicle seat 900 and fastens a seat belt 901. In this state, FIG. 15 shows a case where the user 230 performs mid-air manipulation with fingers 231 on a space-floating image 3 displayed by the space-floating image display device 1000 in front (Y direction). The situation of the user 230 may be either a driver sitting in the driver's seat or a passenger sitting in the passenger seat.

[0240] According to the comparative example of FIG. 15, a space-floating image display device 1000 is fixedly installed in front of a seat 900 of a vehicle. A housing 1190 is fixed to the vehicle. In the illustrated example, the housing 1190 is installed at an angle so that the direction of the optical axis of the space-floating image 3 (z direction) faces a direction (in other words, a reference observation direction) that is assumed to be the position of the viewpoint 232 of the seated user 230 (in other words, a reference observation position). In this case, the imaging position of the space-floating image 3 is also fixed to a predetermined position within the space of the vehicle. Meanwhile, the user 230 seated in the seat 900 can adjust and change the fore-and-aft position and up-and-down position of the seat 900 (fore-and-aft movement 900a, up-and-down movement 900b, and rotational movement 900c of the seat 900). The users 230 vary in height and can move their heads, etc.

[0241] Therefore, depending on the state of the user 230 relative to the seat 900, the floating in space image 3 at a predetermined position may not be accurately / preferably viewed from the position of the viewpoint 232 of the user 230. In other words, depending on the state of the seat 900 and the user 230, the position of the viewpoint 232 and the observation direction may deviate from the standard viewpoint position and observation direction. The standard viewpoint position and observation direction are the viewpoint position, direction, and angle that allow the floating in space image 3 at a predetermined position to be more preferably viewed. The standard observation direction is the direction opposite to the z direction shown in the figure. If the observation direction from the viewpoint 232 of the user 230 (for example, depression angle A1) is within a certain angle range with respect to the z direction, the floating in space image 3 can be accurately / preferably viewed.

[0242] In the third embodiment, a configuration for solving the problem that occurs in the method in which the space floating image display device is fixed to a vehicle or the like, as shown in the comparative example of Fig. 15, is shown. In particular, a configuration for solving the problem that the space floating image 3 cannot be accurately / preferably viewed depending on the state of the seat 900 and the viewpoint 232 of the user 230 is shown.

[0243] [Floating image display inside the vehicle] FIG. 16 is a schematic diagram showing a state in which a space-floating image display device 1000 of Example 3 is installed inside a vehicle 6. The space-floating image display device 1000 is installed on the body or dashboard 7 of the vehicle 6. A user 230, who is an occupant, sits in a seat 900 of the vehicle 6. The user 230 visually recognizes the space-floating image 3 displayed by the space-floating image display device 1000, and can perform mid-air manipulation of the space-floating image 3 with his / her fingers. The in-vehicle system 300 provided in the vehicle 6 has a controller 301, a sensor 302, etc. The sensor 302 includes, for example, an in-vehicle camera, a seat sensor, etc. The space-floating image display device 1000 may communicate with the controller 301, etc., to obtain data and information from the controller 301, etc.

[0244] [Embodiment 3 of the space floating image display device] FIG. 17 shows an example of the configuration of the components of the space-floating image display device 1000 of Example 3. The configuration of FIG. 17 differs from the configuration of FIG. 3 mainly in that it includes a mechanism for adjusting the position and angle (in other words, the direction) of the space-floating image 3, which is the displayed image. Specifically, the configuration of FIG. 17 includes a housing actuator 600, a display unit actuator 700, and an aerial operation detection sensor actuator (sensor actuator) 800. The housing actuator 600 is an actuator that can adjust the angle and direction of the placement of the housing 1190 within the space of the vehicle 6. The display unit actuator 700 is an actuator that can adjust the position of the imaging plane of the space-floating image 3 relative to the housing 1190 by moving the display unit (image display device 1). The sensor actuator 800 is an actuator that can adjust the detection plane (detection light range) of the aerial operation detection sensor 1351 to follow and correspond to the position of the imaging plane of the space-floating image 3.

[0245] The space floating image display device 1000 of the third embodiment adjusts the direction of the housing 1190 (direction 2001 in FIG. 20A described later) by moving the housing actuator 600 through manual operation by the user 230 or automatic control by the control unit 1110 (image processing unit).

[0246] Furthermore, in the third embodiment, the space-floating image display device 1000 may change the direction of the housing 1190 by the housing actuator 600 based on the appropriate emission angle calculated by the control unit 1110 (image processing unit) based on input information obtained from the camera of the imaging unit 1180. The appropriate emission angle here is an angle (angle B in FIG. 21A described later) corresponding to the emission direction 2002 of the image light that corresponds to the observation direction 233A from the viewpoint 232 of the user 230 in FIG. 20A described later in the space inside the vehicle. This provides an emission angle of the imaging light beam of the space-floating image that is suitable for the user 230 to view the space-floating image 3.

[0247] Furthermore, the space-floating image display device 1000 of the third embodiment is configured to move the image display device 1, which is a display unit, and the aerial operation detection sensor 1351 using the display actuator 700 and the sensor actuator 800, based on input information obtained from the imaging unit 1180, the aerial operation detection sensor 1351, etc. Based on the emission angle of the image light from the housing 1190, the display actuator 700 moves the display unit, thereby changing the position of the image formation plane of the space-floating image 3 in the emission direction 2002. Furthermore, the space-floating image display device 1000 moves the aerial operation detection sensor 1351 using the sensor actuator 800 so that the detection plane of the aerial operation detection sensor 1351 follows the position of the image formation plane of the space-floating image 3.

[0248] [Basic operation flow] Fig. 18 shows a basic operation flow (including the operation by the user 230) of the space floating image display device 1000 of the third embodiment. As shown in Fig. 18, the configuration of the third embodiment has four stages of control, roughly divided. The flow of Fig. 18 has a first control of the first stage in step S1, a second control of the second stage in step S2, a third control of the third stage in step S3, and a fourth control of the fourth stage in step S4.

[0249] The first control / operation in the first stage of step S1 is seat position adjustment. This refers to a user 230, for example, a driver, adjusting a seat 900, for example, a driver's seat, to a position suitable for driving, as shown in Fig. 15 (see Fig. 20A described later). This control / operation is a standard control / operation in a general vehicle, and therefore a detailed description thereof will be omitted.

[0250] The second control / operation of the second stage of step S2 is device angle adjustment, which is adjustment of the arrangement of housing 1190 using housing actuator 600, and adjustment of the direction (direction 2001 in FIG. 20A) and emission angle (emission direction 2002 in FIG. 20A) of space floating image display device 1000.

[0251] The third control operation of the third stage of step S3 is a display unit position adjustment. This is an adjustment of the position of the display unit (image display device 1) using the display unit actuator 700, and an accompanying adjustment of the position of the imaging plane of the space floating image 3.

[0252] The fourth control operation of the fourth stage in step S4 is input image display. This is a step in which any display image (content, GUI, etc.) is displayed on the space floating image 3 set up until step S3.

[0253] 18, a first method for adjusting the device angle is manual adjustment by the user 230, and a second method is automatic adjustment by the control unit 1110 (image processing unit). In the case of automatic adjustment, the control unit 1110 (image processing unit) detects the viewpoint position of the user 230 based on the camera of the imaging unit 1180, calculates the appropriate emission direction from the viewpoint position, and drives the housing actuator 600 to align with the appropriate emission direction. Step S2 includes calculation of such viewpoint position, appropriate emission direction, etc.

[0254] In step S3, the user 230 uses the guide on the GUI screen described below to move the operating finger 231, and the display unit actuator 700 is driven to move the display unit in accordance with the tip of the operating finger 231 detected by the aerial operation detection sensor 1351, thereby adjusting the position of the floating image 3 in space.

[0255] In step S4, the display magnification of the image displayed on the screen of the display unit (image display device 1) is adjusted so that the displayed image fits within a display range corresponding to the distance from the viewpoint position (viewpoint distance), which will be described later. This reduces or prevents the screen from being cut off.

[0256] [Device angle adjustment mechanism: Housing actuator] 19A and 19B show the device angle adjustment mechanism, which corresponds to the housing actuator 600.

[0257] 19A, mechanism 1900A has a configuration in which front wheels 612 and rear wheels 613 move on rails 611 (611a, 611b) arranged in an arc shape. Rails 611 (611a, 611b) are provided on the vehicle 6 (e.g., dashboard 7) side along the Y direction, which is the depth direction (front-rear direction). Front wheels 612 (612a, 612b) and rear wheels 613 (613a, 613b) are provided near the four corners of the bottom surface of housing 1190.

[0258] In this mechanism 1900A, when the housing 1190 of the space floating image display device 1000 is movable, it moves (rotates in an arc) on the rails 611 via the front wheels 612 and rear wheels 613, and thereby the emission angle of the imaging light (angle B in FIG. 21A) can be changed to any angle within a predetermined range. When the housing 1190 is fixed, the front wheels 612 and rear wheels 613 are fixed at a desired position.

[0259] Fig. 21A is an explanatory diagram seen from the side corresponding to Fig. 19A. In Fig. 21A, in a coordinate system (X, Y, Z) within the space of the vehicle, the direction in which the housing 1190 is arranged is indicated by direction 2001, using, for example, the planar direction of the transparent member 100, which is the upper surface (in other words, the normal direction to the front surface of the housing 1190), as a reference. Direction 2001 has a housing angle A with respect to the horizontal direction (Y direction). Furthermore, the emission direction of the imaging light (image light) of the space floating image 3 from the transparent member 100 is indicated by emission direction 2002. Emission direction 2002 has an emission angle B with respect to the horizontal direction (Y direction).

[0260] The positional relationship of the image display device 1 and the optical system (for example, the beam splitter 101, which is a polarization separation member, and the retroreflector 2) with respect to the housing 1190 is predetermined. For example, the direction of the optical axis of the image display device 1 with respect to the plane of the transparent member 100 (the y-axis in the figure) and the angle of incidence α, the direction of the optical axis of the retroreflector 2 with respect to the plane of the transparent member 100 (the z-axis in the figure) and the angle of incidence β, the angle of emission θ (θ = A + B) of the imaging light with respect to the plane direction of the transparent member 100, etc. are predetermined. In this example, the angle of emission θ of the imaging light is approximately 45 degrees.

[0261] There is a predetermined relationship between a direction 2001 (casing angle A) of the illustrated casing 1190 and a direction 2002 (emission angle B) of the imaging light. In this example, θ=A+B, B=θ-A. As a result of moving and adjusting the casing 1190 using the casing actuator 600, the direction 2001 (casing angle A) of the casing 1190 is determined, and accordingly, the emission direction 2002 (emission angle B) of the imaging light is also determined.

[0262] State A in FIG. 21A is an example of a state of arrangement (movement, rotation, etc.) of housing 1190 on rail 611, and state B is an example of another state transitioned from state A. In state A, direction 2001 of housing 1190 is directed diagonally downward and forward relative to the horizontal. In state B, direction 2001 of housing 1190 is almost horizontal (Y direction) (housing angle A = 0 degrees). Output angle B corresponding to output direction 2002 is almost the same as predetermined angle θ. Position P1A of the center of floating image 3 in state A and position P1B of the center of floating image 3 in state B are close to each other. Output direction 2002 in state A is closer to the horizontal direction than output direction 2002 in state B, and output angle B has a shallower elevation angle.

[0263] The imaging plane (xy plane) of the space floating image 3 in state A is tilted further forward, towards the user 230, compared to the imaging plane (xy plane) of the space floating image 3 in state B. If the z-axis direction is considered to be the reference observation direction of the user 230, in state A the user 230 views the space floating image 3 at a shallower depression angle, and in state B the user 230 views the space floating image 3 at a deeper depression angle.

[0264] 19A, the movement and change of the housing 1190 on the rail 611 may be performed manually by the user 230, or may be automatically controlled by the space floating image display device 1000. In the former case, the user 230 can adjust the housing 1190 by, for example, pushing or pulling it with his / her hand. In the latter case, the control unit 1110 (image processing unit) may automatically control and drive the housing actuator 600 using input information from the imaging unit 1180, for example.

[0265] 19A has the following advantages. For example, if the center (point P1) of the space floating image 3 is designed to coincide with the center of the arc of the rail 611 (i.e., the track), even if the space floating image display device 1000 is moved on the rail 611, the absolute imaging position of the space floating image 3 will hardly change. Therefore, there is an advantage that it can flexibly respond to the movement of the viewpoint of the user 230 while driving.

[0266] 19B shows another example of the configuration of the device angle adjustment mechanism (casing actuator 600). In FIG. 19B, mechanism 1900B has a configuration in which the entire device (including image display device 1, optical system, and space-floating image 3), in other words, the casing 1190, rotates around a rotation axis 620 fixed to the space-floating image display device 1000. The rotation axis 620 is an axis extending in the X direction, and is disposed near the middle of the casing 1190 in the Y direction. By rotating the space-floating image display device 1000 (casing 1190) around the rotation axis 620, the direction 2001 of the casing 1190 and the emission direction 2002 of the imaging light can be changed arbitrarily within a predetermined range.

[0267] FIG. 21B is an explanatory diagram seen from the side corresponding to FIG. 19B. State A in FIG. 21B is an example of a state in which the housing 1190 is arranged (moved, rotated, etc.) around the rotation axis 620, and state B is an example of another state transitioned from state A. The direction 2001 in state A in FIG. 21B is the same as the direction 2001 in state A in FIG. 21A. State B is a state in which the housing 1190 has rotated from state A by a rotation angle R1 around the rotation axis 620, and the direction 2001 of the housing 1190 is almost horizontal, similar to state B in FIG. 21A. The center position P1A of the floating-in-space image 3 in state A and the center position P1B of the floating-in-space image 3 in state B are slightly separated. The imaging position of the floating-in-space image 3 in state A is closer to the user 230 than the imaging position of the floating-in-space image 3 in state B.

[0268] The rotation (movement / change) of the housing 1190 around the rotation axis 620 in FIG. 19B may be similarly performed manually by the user 230, or may be automatically controlled by the space floating image display device 1000.

[0269] The advantages of the configuration in Fig. 19B include the following: With this configuration, the moving area of ​​the entire device is reduced compared to Fig. 19A, which has the advantage of enabling more effective use of the space inside the vehicle.

[0270] 19A and 19B, known techniques can be applied as technical means for fixing and holding the housing 1190 at the selected position. For example, a brake mechanism for the wheels (612, 613) on the rail 611, a stepping motor for the rotation shaft 620, etc. can be applied.

[0271] [Example 3A: Space-floating image display device] FIG. 20A shows the space-floating image display device 1000 of Example 3A corresponding to the configuration of FIG. 19A, particularly showing a change in position of the display unit and the space-floating image 3. In the space inside a vehicle, the space-floating image display device 1000 is installed in front (Y direction) of a user 230 seated in a seat 900. The housing 1190 is equipped with a housing actuator 600, a display unit actuator 700, and a sensor actuator 800 corresponding to the configuration of FIG. 19A. The optical system applied in FIG. 20A is based on FIG. 2A, but is not limited to this and can also be applied to FIG. 2D. Note that a controller (not shown, such as the control unit 1110 of FIG. 17) may be installed or connected inside or outside the housing 1190 for the image display device 1 and each actuator of FIG. 20A, etc.

[0272] 20A, the embodiment 3A is provided with a housing actuator 600 that can adjust the angle and direction of the space floating image display device 1000 so that the space floating image 3 can be more accurately / preferably viewed depending on the state of the user 230 in the seat 900. The housing actuator 600 has, as components, wheels 612, 613 and a rotating rail 611. The rotating rail 611 is installed on the vehicle side.

[0273] The direction 2001 of the housing 1190 and the emission direction 2002 of the imaging light are changed and adjusted by rotating (in other words, moving on the arc) the entire device (housing 1190) on the arc-shaped rail 611. In other words, the direction 2001 of the housing 1190 and the emission direction 2002 of the imaging light can be set to match the viewpoint position (area 232A, eye box, etc.) and observation direction 233A that are suitable for the user 230, depending on the seating state of the user 230. This allows the floating image 3 to be viewed more accurately and preferably from various possible viewpoint positions of the user 230 seated in the seat 900.

[0274] However, such a configuration that allows the direction of the housing 1190 to be adjusted does not necessarily mean that the floating image 3 can be suitably operated in mid-air according to the position of the user's 230's arm, etc. relative to the seat 900. Because the distance between the floating image 3 and the user's 230 operating finger 231 is not constant according to the fluctuations in the state of the user 230 in the seat 900 and the fluctuations in the direction of the housing 1190 and the floating image 3, it is not necessarily possible to operate the floating image 3 in mid-air at a position that is suitable for the length of the user's 230's arm, etc.

[0275] Therefore, in this embodiment, the display unit (image display device 1) is further moved using the display unit actuator 700, thereby moving the imaging position of the floating image 3 in the depth direction and front-to-back direction (z-axis in the figure) as seen by the user 230, and adjusting it to a position suitable for mid-air operation.

[0276] [Difference in imaging position] Fig. 22 shows an example of differences in the imaging position of the space floating image 3 based on the display unit actuator 700. Fig. 22 also shows an example of differences in the ease of viewing the space floating image 3 from the user 230 and the ease of operating the space floating image 3 in the air with the operating finger 231, depending on the difference in the position of the space floating image 3.

[0277] In state A of Fig. 22, the position of the floating image 3 in space is at position 3x. First, based on the housing actuator 600, the emission direction 2002 of the imaging light is set so that the floating image 3 in space can be viewed favorably from a favorable viewpoint position of the user 230. The distance Lx is the distance on the optical path from a predetermined position (for example, the position on the x-axis in Fig. 20A) to position 3x where the imaging light goes out and forms the floating image 3 in space.

[0278] However, in state A, when the seated user 230 stretches out his / her arm, the tip of the operating finger 231 does not reach the floating-in-space image 3x. Or, even if it does reach, it is difficult because the arm must be fully stretched. The distance Mx is the distance (horizontal length) from the trunk (for example, shoulder) of the user 230 to the tip of the operating finger 231. In such a case, the user 230 is unable or finds it difficult to operate the floating-in-space image 3x in the air with the operating finger 231. Also, in state A, depending on the eyesight of the user 230, the user 230 may find it difficult to view the floating-in-space image 3x.

[0279] State B is another example. The position of the floating-in-space image 3y in state B is closer to the user 230 (closer) on the z-axis than its position in state A. The distance Ly is greater than the distance Lx. However, in state B, although the user 230 can touch the tip of the operating finger 231 to the floating-in-space image 3y, the distance My is relatively short, so the user 230 may find it difficult to operate in the air. Also, in state B, the user 230 may find the floating-in-space image 3y too close and difficult to view.

[0280] As in the above example, the difference lies in whether the floating image 3 can be suitably operated in mid-air (touch operation, etc.) by the operating finger 231 depending on the state of the seat 900 and the user 230, and the state of the user's 230 arm and viewpoint. In this embodiment, in the case of state A, the display actuator 700 can be used to adjust the position where the floating image 3 is formed so that it is closer to the viewer in the z-axis direction. In the case of state B, the display actuator 700 can be used to adjust the position where the floating image 3 is formed so that it is closer to the viewer in the z-axis direction.

[0281] [Display actuator] 20A, as shown at positions 1a and 1b, a display unit actuator 700 is provided that can move the position / state of the display unit (image display device 1) in the y-axis direction shown in the figure. The y-axis direction is the direction in which image light is emitted from the image display device 1 toward the plane of the transparent member 100. This allows the position / state of the floating image 3 in space to be changed in the z-axis direction, as shown at positions 3a and 3b.

[0282] Position 1a is an example of a state in which the display unit is closer to the transparent member 100 on a straight line in the y-axis direction. Position 1b is an example of a state in which the display unit is farther from the transparent member 100 on a straight line in the y-axis direction. Position 1a may be one end of the movable range, and position 1b the other end. Position 3a is an example of a state in which the floating-in-space image 3 is closer to the transparent member 100 (farther back from the user 230) on a straight line in the z-axis direction. Position 3b is an example of a state in which the floating-in-space image 3 is farther from the transparent member 100 (farther forward from the user 230) on a straight line in the z-axis direction. Position 3a may be one end of the movable range, and position 3b the other end.

[0283] State 1a corresponds to state 3a, and state 1b corresponds to state 3b. For example, when the display unit (image display device 1) is positioned at state 1a, the length of the optical path of the image light up to the transparent member 100 (x-axis) becomes shorter accordingly, and correspondingly, the length of the optical path up to the imaging position (state 3b) of the floating in space image 3 by the image light emitted to the outside from the transparent member 100 (x-axis) also becomes shorter.

[0284] The display actuator 700 provided in the housing 1190 is a mechanism that can, for example, support the side portion of the image display device 1 and move (translate) the image display device 1 so as to slide within a predetermined range along the y-axis direction.

[0285] FIG. 23 is a supplementary explanatory diagram of the display actuator 700, and is a schematic xy-plane cross-sectional view of the display unit viewed in the z-axis direction. The image display device 1, which is the display unit, is configured to include, for example, a light source device 13, a liquid crystal display panel 11, which is a display element, and an absorptive polarizer 12. The side surface (at least a portion of the side surface) of the image display device 1 is supported by the display actuator 700, and is capable of sliding in the y-axis direction. The position of the display unit is assumed to be, for example, a position based on the screen of the display element 11. In the display actuator 700, the display unit can be moved and changed, for example, between positions 11a and 11b. When in position 11a, the distance on the optical path from the display unit to a point on the z-axis (transparent member 100) is distance d11a. When in position 11b, the distance on the optical path from the display unit to a point on the z-axis (transparent member 100) is distance d11b. The distance to the imaging position (floating distance / projection distance; Lx, Ly in FIG. 22) is also determined according to this distance.

[0286] [Actuators for sensors] Furthermore, this embodiment is provided with a sensor actuator 800 that can move the aerial operation detection sensor 1351 to follow the displacement of the floating-in-space image 3 based on the display actuator 700, as shown at positions 1351a and 1351b in FIG. 20A . Position 1351a is an example of the position / state of the aerial operation detection sensor 1351, and corresponds to position 3a. Position 1351b is an example of the position / state of the aerial operation detection sensor 1351, and corresponds to position 3b. When in the position 1351a state, the detection surface of the aerial operation detection sensor 1351 (in other words, the range of the detection light) covers the plane of the floating-in-space image 3 in state 3a, as shown by the dashed-dotted line a31. Similarly, when in the position 1351b state, the detection surface of the aerial operation detection sensor 1351 covers the plane of the floating-in-space image 3 in state 3b, as shown by the line a32.

[0287] The sensor actuator 800 provided in the housing 1190 is a mechanism that can, for example, support the side portion of the aerial operation detection sensor 1351 and move (translate) the aerial operation detection sensor 1351 so that it slides within a predetermined range along the depth direction of the housing 1190.

[0288] FIG. 24 is a supplementary explanatory diagram of the sensor actuator 800, and is a schematic XY plane diagram of the area in which the aerial operation detection sensor 1351 is arranged, viewed from the top surface of the housing 1190. An example of the configuration of the aerial operation detection sensor 1351 itself is shown in FIG. 25. The side surface (at least a portion of the side surface) of the aerial operation detection sensor 1351 is supported by the sensor actuator 800, and is capable of sliding movement in the Y direction (direction 2001 of the housing). The positions of the aerial operation detection sensor 1351 (particularly the incident and exit surface of the detection light) in the Y direction are set to positions 24a and 24b as shown in the figure. The aerial operation detection sensor 1351 is capable of moving within the range between positions 24a and 24b.

[0289] As a basic position control, the aerial operation detection sensor 1351 is fixed to a position that has a detection surface that corresponds to the position of the floating image 3 based on the sensor actuator 800. As an applied control, the aerial operation detection sensor 1351 can move its position independently of the position of the floating image 3 based on the sensor actuator 800.

[0290] Fig. 25 shows an example of the configuration of the aerial operation detection sensor 1351. The aerial operation detection unit 1350, control unit 1110, or image control unit 1160 in Fig. 17, in other words the image processing unit, uses the detection signal of the aerial operation detection sensor 1351 to determine and detect aerial operations in the display range 3R of the floating-in-space image 3.

[0291] FIG. 25 shows an xy plane view aligned with the display range 3R of the floating-in-space image 3. The mid-air operation detection sensor 1351 has a plurality of optical elements 2500 arranged in the x direction. The optical elements 2500 are pairs of a light-emitting element 2501 and a light-receiving element 2502. The light-emitting element 2501 is configured, for example, as an infrared element. The light-emitting element 2501 emits light a1, for example, infrared light, in the y direction. If the light a1 is not blocked by an object, it passes through the display range 3R. If the light a1 is blocked by an object, it is reflected by the object and returns as reflected light a2. The reflected light a2 is received by the light-receiving element 2502.

[0292] For example, if a contact point a3 made by the operating finger 231 of the user 230 is within the xy plane of the display range 3R, light a1 is reflected from the contact point a3 and returns as reflected light a2. A light receiving element 2502 at a certain x-direction position detects the reflected light a2. This allows the mid-air operation detection unit 1350 to determine that the contact point a3 is located at that x-direction position. Furthermore, the mid-air operation detection unit 1350 can calculate the distance using the TOF method from the time it takes for the emitted light a1 to return as reflected light a2. For example, it can calculate the distance a4 to the contact point a3. This also allows the position coordinates of the contact point a3 on the xy plane of the display range 3R to be determined.

[0293] Not limited to this example, the aerial operation detection sensor 1351 may be placed above, to the left or right of, or at a position shifted in the z direction, which is the front-to-back direction, with respect to the xy plane of the display range 3R. Multiple aerial operation detection sensors 1351 may be placed at multiple positions in the front-to-back direction.

[0294] [Displacement of each part] 20A, 21A, etc., the predetermined angle formed by the transparent member 100 (direction 2001 of the housing) and the outgoing ray of the imaging light (z-axis, outgoing direction 2002) is defined as angle θ. When the display unit (image display device 1) is moved, for example, from position 1a to position 1b by the display unit actuator 700, the displacement in the y-axis direction is defined as Δ1. Correspondingly, the imaging position of the space-floating image 3 changes from position 3a to position 3b. In this optical system, the image display device 1 and the space-floating image 3 are generated at positions that are plane-symmetrical with respect to the beam splitter 101 and the transparent member 100. In other words, position 3b of the space-floating image 3 is a position that is plane-symmetrical with respect to position 1b of the image display device 1, with the beam splitter 101 and the transparent member 100 as the boundary surface. Therefore, the displacement in the z-axis direction of the space-floating image 3 between positions 3a and 3b is also Δ1.

[0295] On the other hand, the aerial operation detection sensor 1351 for detecting the aerial operation of the floating image 3 in space at position 3b after displacement also needs to move along the direction 2001 (y=(tan θ)×z) of the housing 1190 in order to make the detection surface (the dashed dotted line) follow from position 3a to position 3b. The sensor actuator 800 moves the aerial operation detection sensor 1351 from position 1351a to position 1351b so that this happens. The displacement during this movement is displacement Δ2, which corresponds to displacement Δ1.

[0296] [Example 3B: Space-floating image display device] Fig. 20B is similar, but shows the space floating image display device 1000 of Example 3B corresponding to the configuration of Fig. 19B, particularly showing the position change of the display unit and the space floating image 3. The housing 1190 is equipped with the housing actuator 600 (rotation shaft 620), the display unit actuator 700, and the sensor actuator 800 corresponding to the configuration of Fig. 19B. The functions of the display unit actuator 700 and the sensor actuator 800 are the same as Fig. 20A.

[0297] [Variation: Sensor] The above-mentioned Examples 3A and 3B are examples in which the detection of the aerial operation / input by the operating finger 231 on the floating-in-space image 3 is realized using the aerial operation detection sensor 1351 (e.g., FIG. 25), and a sensor actuator 800 corresponding to that sensor configuration is provided. The detection of the aerial operation / input (in other words, sensing) is not limited to the aerial operation detection sensor 1351, and may be realized using other types of sensors. For example, sensing using the camera of the imaging unit 1180, a capacitance sensor, or the like may be adopted. In this case, a sensor actuator 800 corresponding to that sensor configuration may be used.

[0298] In the case where the space-floating image display device 1000 uses, for example, the imaging unit 1180 (FIG. 20A, etc.) as a sensor, it can detect and determine an operation in the air on the plane of the space-floating image 3 (display range 3R) by estimating the distance between the housing 1190 (space-floating image 3) and the operating finger 231, the position of the operating finger 231, etc. based on image analysis from the image / video captured by the camera. Therefore, it is possible to deal with this by adjusting the setting information of the distance threshold for determining this input (the presence of an operation in the air) in accordance with the adjustment of the imaging position of the space-floating image 3. In other words, in this case, no special hardware is required for the sensor actuator 800 corresponding to the imaging unit 1180, and sensing tailored to the imaging position is possible using software programs and setting information.

[0299] Furthermore, when using a capacitance sensor, the capacitance sensor is installed at a position close to the floating in space image 3. In this case, the method mainly uses the setting of a capacitance threshold to determine the proximity / contact state of the operating finger 231 to the floating in space image 3. In this case, too, it is possible to deal with this by adjusting the setting information of the capacitance threshold for determining that an input (operation in the air) has occurred.

[0300] In addition to these, other implementation examples of the sensor actuator 800 relating to the imaging unit 1180 and the capacitance sensor may include a mechanism that can change the position of the camera or capacitance sensor of the imaging unit 1180 by sliding movement, etc., as in the case of the aerial operation detection sensor 1351.

[0301] [Control example: movement control] 26 shows a configuration example in which an image processing unit such as the control unit 1110 automatically controls each actuator to adjust the position. The space-floating image display device 1000 includes an image processing unit 1001, a display unit 1002, an optical system 1003, a user operation detection mechanism 1004, a memory (storage unit) 1005, a communication unit 1132, an operation input unit 1107, a housing actuator 600, a display unit actuator 700, a sensor actuator 800, and the like. The image processing unit 1001 corresponds to the control unit 1110, the image control unit 1160, the mid-air operation detection unit 1350, and the like in FIG. 17. The display unit 1002 corresponds to the image display device 1 in FIG. 17 or FIG. 20A. The optical system 1003 corresponds to the retroreflector 1101 in FIG. 17, or the beam splitter 101 and retroreflector 2 in FIG. 20A. The user operation detection mechanism 1004 includes an aerial operation detection sensor 1351 and an imaging unit 1180 .

[0302] The image processing unit 1001 automatically performs the above-mentioned position adjustment according to the setting information of the memory 1005, or user instruction input from the communication unit 1132 or operation input unit 1107, or detection of user instruction input to the space floating image 3. The setting information is setting information regarding the position and direction to set the space floating image display device 1000 and the space floating image 3, and setting information regarding the positions to set the housing 1190, the display unit 1002, and the mid-air operation detection sensor 1351, etc. The setting information may be a numerical value for driving each actuator.

[0303] In the third embodiment, the image processing unit 1001 estimates, for example, the viewpoint position and viewpoint distance of the user 230 based on the data of the image captured by the camera of the imaging unit 1180. In addition, in the fourth embodiment described below, the image processing unit 1001 detects the approach of the operating finger 231 to the floating-in-space image 3 (particularly the movable range) based on the data of the image captured by the camera of the imaging unit 1180.

[0304] In the third embodiment, during control, the image processing unit 1001 first controls the housing actuator 600 to move the housing 1190 to a specified position and angle. Next, the image processing unit 1001 controls the display unit actuator 700 to move the display unit 1002 (image display device 1) to a specified position. Then, the image processing unit 1001 controls the sensor actuator 800 to move the aerial operation detection sensor 1351 to a specified position.

[0305] [Position adjustment control and operation] Next, a description will be given of an example of control and operation when adjusting the position of the entire device (space floating image display device 1000 and space floating image 3) in Example 3. User 230 performs the position adjustment work as appropriate.

[0306] (1) In the initial state of the space floating image display device 1000, the direction 2001 of the housing 1190 is set to a certain direction according to the housing actuator 600. For example, the wheels (612, 613) in Fig. 19A are locked so as not to move. In other words, in the initial state, the space floating image 3 is set to a certain tilted state.

[0307] (2) In the initial state of the space-floating image display device 1000, the display unit (image display device 1) is fixed, for example, at a position (e.g., position 1b) farthest from the transparent member 100 (x-axis) within the movable range (position changeable range 2300 in FIG. 23) of the display unit actuator 700. Along with this fixed position (position 1b), the space-floating image 3 is also formed at a position (e.g., position 3b) farthest from the transparent member 100. Also, along with this image formation position, the aerial operation detection sensor 1351 is fixed at position 1351b so that it has a detection surface aligned with this image formation surface.

[0308] (3) The space-floating image display device 1000 detects the presence or approach of the user 230 (for example, a face) in front of the housing 1190 using a camera or the like (which may be a human sensor or the like) of the imaging unit 1180. At the timing of detection, the space-floating image display device 1000 displays the image formation plane position adjustment introduction screen 4000 (FIG. 27A) as a GUI screen using the space-floating image 3.

[0309] Moreover, state A in FIG. 28 shows states corresponding to the above (1), (2), and (3).

[0310] [GUI screen example] 27A and 27B show an example of a GUI screen based on the space floating image 3. This screen is a GUI screen that guides and directs the user 230 to more easily adjust the position of the imaging plane of the space floating image 3.

[0311] Screen 4000 in Fig. 27A shows an image plane position adjustment introduction screen 4000. On this screen 4000, a guide message such as "We will adjust the imaging position of the aerial image. Please extend your arm towards the image to a position that makes it easy to operate" is displayed. Also, at a predetermined position on this screen 4000, an image area 4000a is displayed as a guide for user 230 to extend their arm (operating finger 231). In this example, a white circle is displayed near the center of screen 4000 as image area 4000a.

[0312] (4) Following the instructions on the screen 4000, the user 230 moves the operating finger 231 toward a region where the arm can be naturally extended, a position further back than the imaging plane of the floating image 3 (corresponding image region 4000a), and stops it at the desired position.

[0313] State B in Fig. 28 shows a state corresponding to (4) above. Position 2801 shows the position of the tip of the operating finger 231, in particular the position that matches the tilt of the floating-in-space image 3. In state B, the operating finger 231 is at position 2801, which is slightly further back than the floating-in-space image 3 (corresponding image area 4000a) at the forefront position 3b.

[0314] (5) The space-floating image display device 1000 obtains an input / detection signal from the mid-air operation detection sensor 1351 in response to the action of the user 230, that is, the contact of the operating finger 231 with the image area 4000a of the space-floating image 3 at position 3b. Then, the space-floating image display device 1000 displays the imaging position adjustment screen 4001 (FIG. 27A).

[0315] 27A shows an imaging position adjustment screen 4001. On this screen 4001, a guide message such as "The imaging position of the aerial image is being adjusted. Please do not move your arms" is displayed.

[0316] (6) While the floating-in-space image display device 1000 is detecting an input by the aerial operation detection sensor 1351, i.e., a state in which the operating finger 231 is in contact with the floating-in-space image 3, the sensor actuator 800 controls the position of the aerial operation detection sensor 1351 to move from position 1351b in a direction in which the z coordinate decreases (a direction in which the beam splitter 101 approaches the x-axis). This movement is continuous and at a constant speed. The displacement of the position of the aerial operation detection sensor 1351 is defined as Δ3.

[0317] (7) Furthermore, the space-floating image display device 1000 moves the space-floating image 3 along the sensor detection surface (dash-dotted line) in response to the displacement Δ3 of the aerial operation detection sensor 1351. To do this, the space-floating image display device 1000 controls the display unit actuator 700 to move the display unit (image display device 1) from position 1b in the direction (+y) that brings it closer to the transparent member 100 (x-axis) on the y-axis by a displacement amount Δ4 (Δ4=Δ3×cosθ) that corresponds to the displacement Δ3.

[0318] State C in Figure 29 shows states corresponding to the above (6) and (7). Position 2901 shows the desired position where the user 230 stops the tip of the operating finger 231. Position 1351c is a position corresponding to displacement Δ3. Position 1351c is a position corresponding to displacement Δ4.

[0319] (8) If the above-mentioned operation of following the operating finger 231 continues, the input / detection from the mid-air operation detection sensor 1351 will cease (become no detection) near the tip of the stationary operating finger 231. Therefore, the space floating image display device 1000 determines that the position at this time is the mid-air operation area suitable for the user 230, ends the operation of the sensor actuator 800 to displace the mid-air operation detection sensor 1351, and the display unit actuator 700 to displace the display unit, and fixes it in a position that matches the mid-air operation area.

[0320] State D in Fig. 29 shows a state corresponding to (8) above. The display unit is at position 1c, and the imaging plane of the floating image 3 is set at position 3c corresponding to position 2901. In other words, the positions after adjustment that are suited to the preferred position of the arm and operating finger 231 of the user 230 are positions 1c and 3c.

[0321] After adjusting the position as in the above example, the space-floating image display device 1000 displays a screen to confirm with the user 230 whether it is OK to complete the position adjustment. In this example, the space-floating image display device 1000 displays an imaging position adjustment screen 4002 as shown in FIG. 27B on the space-floating image 3. On the screen 4002, a guide message such as "Adjustment of the imaging position of the aerial image completed. Fixing it at the current position. Are you sure?" is displayed. The user 230 looks at the screen 4002 and presses the "OK" (end) button if there is no problem with the current position after adjustment, or presses the "NG" (readjust) button if they want to readjust it.

[0322] When "OK" is input, the space-floating image display device 1000 saves the setting information, ends the adjustment mode, enters normal display mode, and displays the specified image content. When "NG" is input, the space-floating image display device 1000 resets to the default position (for example, the initial position of state A in FIG. 28), and performs readjustment from that position using the same procedure.

[0323] The space-floating image display device 1000 registers and saves information on the positions of each part set in the above procedure (such as the coordinates of each actuator) as setting information in a non-volatile memory, etc. This allows the space-floating image display device 1000 to reproduce the set positions as default positions by reading the setting information from the memory the next time it is started up.

[0324] Furthermore, the setting information such as the position of the entire device may be a single setting information for this device that does not depend on the user 230, or may be different setting information for each user 230. In the latter case, the user 230 may be recognized by facial recognition using the imaging unit 1180, and the user information may be associated with the setting information such as the position and registered. Furthermore, it may be possible to set and store multiple patterns of setting information for each user 230 so that multiple patterns of position setting can be used for each user 230. For example, a first pattern of position setting information that is suitable for driving and a second pattern of position setting information that is suitable for non-driving may be provided.

[0325] In the above example, the position of the display unit and the sensor are adjusted after adjusting the direction and position of the housing 1190, but this is not limiting. After adjusting the positions of the display unit and the sensor as in the above example, it is also possible to adjust the direction and position of the housing 1190 again using the housing actuator 600.

[0326] [Variation: Guidance Target] During the control and operation of the position adjustment, in the example of the screen 4000 (FIG. 27A), an image area (in other words, a guidance target) 4000a is provided so as to guide the position of the detection point of the operating finger 231 to the center of the imaging surface (display range 3R) of the space-floating image 3. This is not limiting. The imaging surface (display range 3R) of the space-floating image 3 can be tilted in various ways depending on the direction of the housing 1190. The imaging surface does not necessarily face the arm and operating finger 231 of the user 230 directly. Therefore, for example, as in FIGS. 28 and 29, when the imaging surface of the space-floating image 3 is tilted obliquely with respect to the horizontal, the upper part of the imaging surface (display range 3R) of the space-floating image 3 becomes a point that is relatively far from the arm and operating finger 231, and the lower part becomes a point that is relatively near.

[0327] Using this, as a modified example, a screen may be provided with an image area 4000b above the image formation surface (display range 3R) of the floating image 3 in space so as to guide the arm and operating finger 231.

[0328] This modification is shown in Fig. 30. The space-floating image display device 1000 displays a screen 4000B on the space-floating image 3 at position 3b. This screen 4000B has an image area 4000b for guidance at the top. This allows the operating finger 231 to be guided further back in the z-axis direction. When operating the space-floating image 3, which is set at an angle to the vertical as seen from the user 230, with the operating finger 231, it is ensured that the operation in the air is easy, especially for the upper part of the screen.

[0329] [Variation: Offset adjustment] In the above example, the position of the imaging plane and the position of the display unit are set in accordance with the point near the tip of the stationary operating finger 231 (state D in FIG. 29), that is, the point at which the sensor input / detection by the mid-air operation detection sensor 1351 is discontinued and the point at which the presence or absence of fingertip detection switches, but the present invention is not limited to this, and the following variants are also possible.

[0330] FIG. 31 shows a modified example. The position when sensor input / detection ceases may be used as a reference, and a position offset by a predetermined distance in front or behind this reference position may be used as the setting position for the imaging plane. In FIG. 31, position 3101 is the position when sensor input / detection ceases, and this is reference position A. Positions B and C are positions located a predetermined distance (offset distance: ±g1) in front of and behind position 3101(A) in the z-axis direction. The space floating image display device 1000 sets position B or position C as the position of the imaging plane and sensor detection plane by fine-tuning each actuator. The offset distance ±g1 is prepared as a preset value in advance.

[0331] [Variation: Relative Adjustment] As a modified example, if the user 230 has once set the coordinates of each actuator (the position of each part), when the position or state of the seat 900 is changed, it is possible to use an automatically calculated result to make the actuator coordinates follow, so that the relative operating positions do not change. This will be explained below.

[0332] [Viewpoint tracking adjustment flow] As described above, the space-floating image display device 1000 automatically sets previously set position coordinates of the housing 1190, display unit, imaging surface, and aerial operation detection sensor 1351 as defaults for each user 230 in association with the user information. However, if the user 230 changes the position or state of the seat 900, that is, if the viewpoint position or the like changes, the relative positional relationship between the new viewpoint position and the space-floating image 3 will fluctuate in the default set position coordinates (for example, the distance from the space-floating image 3 will increase). Therefore, the default set position coordinates may no longer be an operation position suitable for aerial operation.

[0333] When using the user setting function, the space floating image display device 1000 performs face recognition of the user 230 using the imaging unit 1180, and accesses setting information (registered parameter values) associated with the identification information of the user 230. The parameters managed in the setting information include, for example, the position coordinates of the space floating image 3 based on the position coordinates of the display unit (which may be the parameter values ​​of the corresponding display unit actuator 700), and the distance from the camera of the imaging unit 1180 to the viewpoint position coordinates of the user 230.

[0334] 32 shows a control flow relating to the modified example, and includes a normal adjustment flow F1 and a viewpoint tracking adjustment flow F2. In step S201, the space-floating image display device 1000 performs face recognition of the user 230 using the imaging unit 1180, identifies the user 230, and refers to the user setting information of the user 230, i.e., the most recent (previous) set position coordinates, etc. In step S202, the space-floating image display device 1000 checks whether there are already registered position coordinates corresponding to the user 230 (whether the position of the space-floating image 3 has already been set according to the seat 900, etc.). If there are no already registered position coordinates, it means that a new registration is required, and the process proceeds to the normal adjustment flow F1. If there are, the function of relative adjustment (in other words, viewpoint tracking adjustment) in the modified example is available, so the process proceeds to the viewpoint tracking adjustment flow F2.

[0335] In the case of the normal adjustment flow F1, for example, the position adjustment described above in Fig. 28 etc. is performed. The normal adjustment flow F1 includes device angle adjustment in step S11, display unit position adjustment in step S12, input video display in step S13, and setting coordinate storage in step S14.

[0336] On the other hand, in the case of the viewpoint tracking adjustment flow F2, automatic position adjustment is performed so that the distance from the camera to the viewpoint position coordinates of the user 230 is always constant in accordance with the movement and change of the position and state (particularly the viewpoint position) of the user 230 relative to the seat 900, in other words, so that the relative positional relationship between the viewpoint position and the imaging surface in the existing settings of the user 230 is the same.

[0337] The viewpoint tracking adjustment flow F2 includes coordinate position estimation in step S21, difference parameter calculation in step S22, device angle adjustment in step S23, display unit position adjustment in step S24, input video display in step S25, and set coordinate storage in step S26.

[0338] In both the normal adjustment flow F1 and the viewpoint tracking adjustment flow F2, the adjusted position coordinates are stored in memory in association with the face authentication information (i.e., user information). The storage timing is after the input video display (steps S13 and S25), but it may be before.

[0339] FIG. 33 is an explanatory diagram regarding the above-mentioned viewpoint tracking adjustment. State A is an example of setting a position for a certain user 230 (user A). In this example, the position 3c of the imaging plane of the space-floating image 3, which the user 230 (A) set to be a suitable operation area according to the suitable seat 900 and viewpoint position E1, is a position near the middle within the range from the furthest position 3a to the foremost position 3b. The viewpoint position E1 has position coordinates (x1, y1, z1) as seen from the space-floating image display device 1000 (for example, a point on the x-axis). The position set in state A (position 3c, etc.) is set as the default position for the user 230, and the setting information is saved.

[0340] State B is an example in which the same user 230 (A) subsequently changes the state of the seat 900 and the viewpoint position (assumed to be E2). In this example, the viewpoint position E2 after the change is closer to the housing 1190 than the viewpoint position E1 in the past state A. In the case of viewpoint tracking adjustment flow F2, the space floating image display device 1000 detects and calculates the change in the viewpoint position as described above based on the camera.

[0341] The distance from the camera of the imaging unit 1180 to the viewpoint position can be converted appropriately based on the positional relationship. For example, the distance D1 to the viewpoint position E1 can be converted to the distance from the x-axis point of the housing 1190. The distance D2 to the viewpoint position E2 can be converted to the distance from the x-axis point of the housing 1190. The space floating image display device 1000 calculates the difference (referred to as DD) between the distance D1 to the viewpoint position E1 and the distance D2 to the viewpoint position E2 in the difference parameter calculation of step S22. In other words, the relative amount of change (DD) of the viewpoint position is calculated.

[0342] In step S22, the space-floating image display device 1000 grasps the amount of change in the vertical and horizontal directions of the current viewpoint position relative to the previous viewpoint position of the user 230. The space-floating image display device 1000 calculates the difference amount (for example, the actuator control amount) to be moved relative to the previous position of the imaging plane from the amount of change in the viewpoint position.

[0343] The space-floating image display device 1000 corrects the actually applied position of the space-floating image 3 from position 3c in the existing (state A) setting information according to the amount of variation (DD). In this example, it is corrected from position 3c to position 3d, which is further back. The space-floating image display device 1000 controls the display actuator 700 to display the space-floating image 3 at position 3d, and controls the sensor actuator 800 to move the aerial operation detection sensor 1351 to the corresponding position 1351d.

[0344] It should be noted that the method of determining the viewpoint position and viewpoint distance is not limited to using the camera of the imaging unit 1180 or image analysis. For example, if information on the settings and status of the seat 900, etc., or information on the viewpoint position from an in-vehicle camera can be obtained from the in-vehicle system 300 (FIG. 16), such information may be used.

[0345] The space floating image display device 1000 may perform the calculations related to the viewpoint tracking adjustment as described above in real time, or may calculate the correspondence between the viewpoint position fluctuation and the actuator control amount in advance, set and store it in a correspondence table, and omit the calculations by referring to the correspondence table.

[0346] When adjusting the viewpoint tracking, the procedure of aligning with the operating finger 231 in the normal adjustment flow F1 (see FIG. 28, etc.) can be omitted. After adjustment to position 3d, the preferred position of the operating finger 231 when the user 230 naturally extends their arm will be a position near position 3d. Comparing state A and state B, the positional relationship from the viewpoint position of the user 230 to the operating finger and the floating-in-space image is roughly the same.

[0347] For a certain user 230, if the state of the user 230 in the seat 900 changes from a state in which the position of the floating image 3 in space has already been set, it is possible to perform the normal position adjustment again in the same way, but as in the above example, it is also possible to omit the normal procedure and perform a relative adjustment from the state that has already been set.

[0348] [Viewable area] Next, an example of control of input video display (step S4 in FIG. 18) will be described.

[0349] Fig. 34 is an explanatory diagram regarding the visible area of ​​the space floating image 3. Here, as shown in Fig. 34, the distance from the position of the space floating image display device 1000, for example, the point on the x-axis of the transparent member 100 (the position where the chief ray of the image light passes, which is assumed to be the origin O) to the position of the viewpoint 232 of the user 230 (which is assumed to be viewpoint position E) is assumed to be the visible distance d. Let us consider the visible area of ​​the space floating image 3 relative to the visible distance d. In other words, viewpoint position E is the observation reference point.

[0350] Image light a1 having a predetermined polarization corresponding to a display image 3010 on the display unit (image display device 1) is reflected by the beam splitter 101, the reflected image light is retroreflected by the retroreflective member 2, and the retroreflected and polarization-converted image light passes through the beam splitter 101 (origin O) and is emitted to the outside. The emitted image light a2 is imaged as a space-floating image 3020, which is a real image, at a predetermined position in space according to the imaging principle described above. The space-floating image 3020 is an image / imaging surface having a rectangular display range 3R (corresponding xy plane) corresponding to the rectangular screen of the display unit (image display device 1).

[0351] In order for this floating image in space 3020 to be accurately / preferably viewed by the eyes (viewpoint 232) of the user 230, the retroreflective member 2 must be present behind the imaging plane (z-axis direction, -z).

[0352] A quadrangular pyramid 3400 formed by connecting the viewpoint 232 (viewpoint position E) of the user 230 with the four corners of the rectangular retroreflective member 2 is similar to a quadrangular pyramid 3401 formed by connecting the viewpoint position E of the user 230 with the four corners of the rectangle of the floating image 3 in space.

[0353] The distance between the beam splitter 101 and the retroreflective member 2 in the z-axis direction, in other words, the distance between the origin O and point Q (the point where the chief ray enters the retroreflective member 2, the center point), is defined as d0 (-d0 in terms of coordinates). The distance between the beam splitter 101 and the space-floating image 3020, in other words, the distance between the origin O and point P (the point where the chief ray passes through the space-floating image 3020, the center point), is defined as d1 (+d1 in terms of coordinates). Then, the similarity ratio of these quadrangular pyramids can be expressed as (d+d0):(d-d1).

[0354] The width of the retroreflective member 2 in the x direction is wx, and the width in the y direction is wy. Then, the width in the x direction (denoted as hx) that can be seen as the floating image 3 in space from the viewpoint E is {(d-d1) / (d+d0)}×wx, and the width in the x direction (denoted as hy) is {(d-d1) / (d+d0)}×wy.

[0355] In the illustrated example, the range defined by widths hx and hy (display range 3R) is smaller than rectangular display image 3020, which corresponds to the entire original display image 3010. Therefore, the entire display image 3020 cannot be viewed from viewpoint E. In other words, display image 3020 has a region that cannot be viewed from viewpoint E, a cut-off region 3020b. Region 3020b is an area that is outside the range defined by widths hx and hy (display range 3R).

[0356] Image 3020 is a conceptual image of the imaging area of ​​the image light of the space floating image 3, which corresponds to the case where image 3010 is displayed at its maximum size on the physical screen of the liquid crystal display panel of the display unit. The entire area of ​​this image 3020 can only be seen from a viewpoint that is farther away on the z axis than the illustrated viewpoint E. In other words, from the illustrated viewpoint E, the screen (display content) is cut off.

[0357] [How the floating images appear] 35A shows how the display screen of the floating-in-space image 3 (the screen corresponding to the displayed image 3010) looks from the viewpoint position E (FIG. 34) of the user 230 who is in the correct position (for example, the position where the seat 900, etc. is adjusted in FIG. 20A, etc.) when the display unit (image display device 1) is in each of the positions described above. Based on the imaging principle of the floating-in-space image display device 1000 described above, the user 230 can only see the area where the retroreflective member 2 is visible behind (-z) the viewpoint position E as the floating-in-space image 3.

[0358] State A is when the display unit is at position 1a, in other words, when the space-floating image 3 is at position 3a. At this time, the entire screen area of ​​the space-floating image 3 (the area corresponding to the display image 3020) is originally area 3000a. However, the area that the user 230 can actually view as the space-floating image 3 is limited to area (visible area) 3001a. Area 3001a is the area corresponding to the range (display range 3R) defined by widths hx and hy in FIG. 34. Therefore, the screen that is actually displayed as the space-floating image 3 looks like screen 3002a corresponding to area 3001a. In other words, the content of the original screen of display image 3010 may be cut off. In the example shown, the screen contains character 3011a and button 3012a as content / GUI. Part of button 3012a is cut off.

[0359] Moreover, state B is when the display unit moves to position 1b, in other words, when the floating-in-space image 3 moves to position 3b. At this time, the imaging distance (d1) increases, so the distance (d-d1) to the viewpoint position E decreases, and the imaging surface moves closer to the viewpoint 232. In this case, compared to when at position 1a, the user 230 sees an even more enlarged image. Therefore, the area that can actually be viewed as the floating-in-space image 3 becomes area (visible area) 3001b, and the content becomes even more noticeably cut off, as shown on screen 3002b. Parts of the character 3011b and button 3012b are cut off.

[0360] Whether the imaging position of the space floating image 3 is the rear position 3a or the front position 3b, the screen (image content) will be cut off to some extent as in the example of FIG. 35A.

[0361] [Screen display magnification control] Therefore, in this embodiment (step S4), in addition to controlling the position adjustment of the display unit as described above, the display magnification of the screen display of the display unit is controlled according to the imaging distance (d1) and the viewpoint position E, as shown in Figure 35B, thereby suppressing and preventing content from being cut off.

[0362] Consider the case when the display unit (image display device 1) is at position 1a (Fig. 20A, etc.). Position 1a is the position closest to the x-axis (origin O) (one end of the movable range). At this time, the spatial floating image 3 is at position 3a (Fig. 20, etc.), and let z = d1. Position 3a is the position closest to the x-axis (origin O) (one end of the movable range). The protruding distance from the origin O to the imaging surface is d1.

[0363] The width in the x-direction and the width in the y-direction of the visible region 3001a can be represented by hx and hy described above (the formula in Fig. 34). Let the display area of the physical screen of the liquid crystal display panel of the display unit be lx×ly using the horizontal width lx and the vertical width ly (Fig. 34).

[0364] Then, when lx < wx and ly < wy, the spatial floating image display device 1000 controls the display magnification so that the display area of the video content (video 3010) on the screen of the liquid crystal display panel matches hx×hy. As a result, in the case of position a1, like state A in Fig. 35B, the spatial floating image 3 visible to the user 230 becomes like the screen 3003a adapted to the region 3001a, and it is possible to reduce and prevent the break of the video content. Similarly, in the case of position 1b, like state B in Fig. 35B, the spatial floating image 3 visible to the user 230 becomes like the screen 3003b adapted to the region 3001b, and it is possible to reduce and prevent the break of the video content. All of the character 3011b and the button 3012b are displayed.

[0365] Fig. 36 shows, as a supplement, the change in the display area of the video by controlling the display magnification on the physical screen of the liquid crystal display panel 11 of the display unit (image display device 1). State A is the case where the video (video content) 3010 is displayed on the physical screen 11g (size lx×ly) of the liquid crystal display panel 11 of the display unit (image display device 1) at position 1b, for example, with the maximum size adapted to the size of the screen 11g.

[0366] State B is a case where, when displaying image 3010 on screen 11g, the display magnification of image 3010 is controlled to produce image 3010b so that image 3010 fits within display area 11gd corresponding to area (visible area) 3001b in FIGS. 35A and 35B. In other words, image 3010b is a reduced image of image 3010. If the width of display area 11gd is (bx, by), the display magnification is (bx / lx, by / ly). Image 3010b is displayed aligned with the center of screen 11g, and the area of ​​screen 11g other than image 3010b is displayed in black (non-display). This allows screen 3003b of floating-in-space image 3 to be displayed as in state B of FIG. 35B.

[0367] [Effects, etc.] As described above, according to the third embodiment, the position and orientation of the floating in space image 3 can be adjusted according to the state of the seat 900 and the viewpoint 232 of the user 230, allowing the user 230 to view the floating in space image 3 appropriately and accurately, and to perform suitable and accurate mid-air operations on the floating in space image 3. Depending on the viewpoint position and viewpoint distance of the user 230, it is possible to adjust and calibrate the position etc. of the floating in space image 3, and it is also possible to suppress and prevent clipping of the screen.

[0368] [Variations] The following modification of the third embodiment is also possible.

[0369] Fig. 20C shows a configuration in which the optical system to be applied is an optical system having the retroreflector 5 of Fig. 2D. This type of optical system can also achieve the same function.

[0370] As another modified example, the housing 1190 may be provided with hardware such as buttons for adjusting the position of the floating image 3 in space. Fig. 37 shows a configuration example of a modified example, seen from the front of the user 230, looking at the vicinity of the top surface of the housing 1190. For example, an operation input unit 3700 is provided near the front of the top surface of the housing 1190. The operation input unit 3700 includes an actuator activation switch 3701, a forward movement button 3702, a backward movement button 3703, and the like. The operation input unit 3700 may be located on the side of the housing 1190, for example.

[0371] First, the actuator activation switch 3701 is a button that can manually turn on (enable) / off (disable) the operation of each actuator in the third embodiment, i.e., the function of position adjustment using the actuator. When the actuator activation switch 3701 is in the off state, the activation (operation) of each actuator is disabled. That is, the housing 1190, the display unit, and the sensor are fixed in their set positions and cannot be moved. When the actuator activation switch 3701 is in the on state, the activation (operation) of each actuator is enabled. That is, the housing 1190, the display unit, and the sensor can be moved based on manual operation or automatic control.

[0372] Furthermore, in this example, the user 230 can manually operate the forward movement button 3702 and the backward movement button 3703 to vary the position of the floating-in-space image 3 forward or backward. This operation can be used independently of the position adjustment function of the third embodiment. When the user 230 continues to press the forward movement button 3702 while the actuator activation switch 3701 is in the on state, the floating-in-space image display device 1000 moves the position of the floating-in-space image 3 forward (+z), for example, by controlling the display unit actuator 700. Furthermore, the floating-in-space image display device 1000 moves the mid-air operation detection sensor 1351 to follow the movement of the floating-in-space image 3 by the display unit. The floating-in-space image display device 1000 stops the floating-in-space image 3 and the mid-air operation detection sensor 1351 at the position when the user 230 releases the forward movement button 3702. Similarly, when the user 230 presses the back movement button 3703, the positions of the floating image 3 and the mid-air operation detection sensor 1351 can be moved to the back side (-z) by controlling each actuator.

[0373] For example, the floating image display device 1000 automatically adjusts the position of the floating image 3 in accordance with the state of the seat 900, etc., and then the user 230 can manually operate the above button to fine-tune the position.

[0374] 28 and the like, the initial position at the start of adjustment of the imaging surface is set to position 3b (the frontmost position within the movable range), and the imaging surface is moved in the direction (-z) toward position 3a (the rearmost position within the movable range) to set the imaging surface at a detection position near the fingertip (referred to as the first method). However, the present invention is not limited to this, and the following method is also possible as a modified example.

[0375] As a second method, the initial position at the start of adjustment of the imaging surface may be position 3a (the furthest position), and the imaging surface may be moved in the direction (+z) toward position 3b (the furthest position) to set the imaging surface at a detection position near the fingertip.

[0376] Fig. 38 is an explanatory diagram of a second method, which is a modified example. The space-floating image display device 1000 displays a guide screen 4000 on the space-floating image 3 at position 3a. The user is asked to extend their arm and operating finger 231 toward the guide area to a suitable position (for example, position 3c). The space-floating image display device 1000 moves the mid-air operation detection sensor 1351 from position 1351a to position 1351b, and detects position 3c where detection of the fingertip changes from absent to present. The space-floating image display device 1000 moves the display unit from position 1a to position 1c so as to set the imaging plane at position 3c.

[0377] In another modified example (referred to as a third method), the initial position of the floating-in-space image 3 at the start of adjustment is set to an arbitrary position that has already been set for each user 230. The user 230 is asked to place the operating finger 231 in a suitable position relative to that initial position based on a guide. The floating-in-space image display device 1000 moves the mid-air operation detection sensor 1351 forward (+z) or backward (-z) from that initial position, detects the position where the presence or absence of detection of the fingertip changes as the position of the tip of the operating finger 231, and sets the imaging plane at that position.

[0378] FIG. 39 is an explanatory diagram of a third method, which is a modified example. For example, suppose that the position of the floating-in-space image 3 set for a certain user 230 is at position 3c, near the middle of the range from position 3a to position 3b. Adjustment is started with this position 3c as the initial position. The floating-in-space image display device 1000 displays a guide screen 4000 on the floating-in-space image 3 at position 3c, and asks the user 230 to place the operating finger 231 in a suitable position. First, if there is no detection by the mid-air operation detection sensor 1351 at position 3c, the floating-in-space image display device 1000 can infer that the operating finger 231 is placed closer to the user than position 3c. Therefore, the floating-in-space image display device 1000 moves the mid-air operation detection sensor 1351 from position 1351c corresponding to position 3c toward the user (+z) toward position 1351b corresponding to position 3b. The space floating image display device 1000 detects the position where the detection of the fingertip changes from absent to present, and sets the imaging plane at that position.

[0379] On the other hand, if there is detection by the aerial operation detection sensor 1351 at position 3c, the space-floating image display device 1000 can estimate that the operating finger 231 is placed further back than position 3c. Therefore, the space-floating image display device 1000 moves the aerial operation detection sensor 1351 from position 1351c corresponding to position 3c to the back side (-z) towards position 1351a corresponding to position 3a. The space-floating image display device 1000 detects the position where the detection of the fingertip changes from present to absent, and sets the imaging plane at that position.

[0380] Example 4 Example 4 will be described with reference to Fig. 40 etc. Example 3 has a function of adjusting the position etc. of the floating image 3 using the actuator 700 for the display part etc., whereas Example 4 has a function of interactively moving and reacting the floating image 3 to the operating finger using the actuator 700 for the display part etc.

[0381] [Fourth embodiment of the space-floating image display device] FIG. 40 shows a configuration example of a space-floating image display device 1000 of Example 4. In FIG. 40, the configuration does not include the housing actuator 600, as compared with the configuration of FIG. 20A etc. of Example 3. Since Example 4 does not focus on adjusting the position, etc., as in Example 3, the housing actuator 600 is not provided. The hardware of the display unit actuator 700 and the sensor actuator 800 in FIG. 40 has the same configuration as Example 3. The method of controlling the actuators in Example 4 is different from Example 3. The usage environment of the space-floating image display device 1000 of Example 4 is not limited to the vehicle 6 (FIG. 16) of Example 3. In the example of FIG. 40, the space-floating image display device 1000 is installed in any environment (for example, a store, a public space, a home, etc.).

[0382] In the fourth embodiment, the floating distance (protrusion distance, position of the imaging plane) of the floating in space image 3 is interactively changed in synchronization with the action of the user 230 bringing the operating finger 231 closer to the floating in space image 3. In the fourth embodiment, the floating in space image 3 is stopped at the position where the moving operating finger 231 contacts the moving floating in space image 3.

[0383] In the fourth embodiment, a display unit actuator 700 is disposed at a position that does not obstruct the imaging light beam of the floating-in-space image 3, and moves the display unit (image display device 1) in the light beam emission direction (y-axis). When the display unit is at position 1a (FIG. 40), the floating-in-space image 3 is imaged at position 3a with a floating distance of d1, and when the display unit is at position 1b, the floating-in-space image 3 is imaged at position 3b with a floating distance of d2. In this embodiment, the movable range of the display unit is from position 1a to position 1b. The movable range of the floating-in-space image 3 is from position 3a to position 3b. The movable range of the mid-air operation detection sensor 1351 is from position 1351a to position 1351b. Position 3a is the first position, which is the rear position on the z-axis where the floating distance is shortest, and position 3b is the second position, which is the front position on the z-axis where the floating distance is longest.

[0384] The space floating image display device 1000 of the fourth embodiment may include an actuator activation switch 4101 or the like in an operation input unit 4100 that is exposed on the top surface of the housing 1190. The user 230 can manually turn on / off the actuator activation switch 4101.

[0385] 41 shows a configuration example of the operation input unit 4100 on the top surface of the housing 1190 as seen from the user 230 in front of the device. In this example, the operation input unit 4100 has an actuator activation switch 4101. The actuator activation switch 4101 is a button that can turn on (enable) / off (disable) the activation (operation) of each actuator in Example 4. In other words, the actuator activation switch 4101 is a button that can turn on (enable) / off (disable) the function of interactively moving the floating image 3 in space using the actuator.

[0386] When the actuator activation switch 4101 is in the OFF state, the display unit actuator 700 and the sensor actuator 800 are controlled so that they cannot be activated (operated). That is, in the OFF state, the display unit (image display device 1) cannot be moved by the display unit actuator 700, and is fixed at a certain preset position. Correspondingly, the floating in space image 3 cannot be moved, and is fixed at a certain preset position. Furthermore, the mid-air operation detection sensor 1351 is also basically fixed at a position that has a detection surface corresponding to the position of the floating in space image 3.

[0387] When the actuator activation switch 4101 is in the on state, it is controlled so that activation (operation) of each actuator is permitted. That is, in the on state, the position of the display unit can be changed by the display unit actuator 700, and the position of the floating image 3 in space can be changed accordingly. Also, the mid-air operation detection sensor 1351 can basically be changed to a position where it has a detection surface that corresponds to the position of the floating image 3 in space.

[0388] When the actuator activation switch 4101 is in the off state, the function of interactively moving the floating in space image 3 in Example 4 is disabled. In the off state, while the floating in space image 3 is set and fixed to a certain position, for example, position 3a, the mid-air operation detection sensor 1351 is fixed to position 1351a corresponding to position 3a based on the sensor actuator 800. The mid-air operation detection sensor 1351 at position 1351a can detect mid-air operations by the operating finger 231 on the screen of the floating in space image 3 at position 3a.

[0389] Furthermore, when the actuator activation switch 4101 is in the on state, the function of interactively moving the floating-in-space image 3 in the fourth embodiment is enabled. In the on state, when the floating-in-space image 3 is in a certain position, for example, position 3a, the floating-in-space image 3 is moved in a direction (+z) approaching the operating finger 231 based on the display actuator 700 in response to detection of the approach of the operating finger 231. At the same time, the mid-air operation detection sensor 1351 is also moved in a direction (+z) approaching the operating finger 231 based on the sensor actuator 800. The mid-air operation detection sensor 1351 detects the contact of the operating finger 231 with the floating-in-space image 3. The floating-in-space image display device 1000 stops the floating-in-space image 3 at the position when the operating finger 231 contacted the floating-in-space image 3. The mid-air operation detection sensor 1351 at a position corresponding to this stopped position can detect the mid-air operation by the operating finger 231 on the screen of the floating-in-space image 3.

[0390] When the user 230 wants to fix the position of the floating image 3 in a predetermined position (initial position), in other words when the user wants to turn off the interactive function, the user 230 turns off the actuator activation switch 4101. When the user 230 wants to turn on the interactive function, the user 230 turns on the actuator activation switch 4101. The predetermined position (initial position) of the floating image 3 in space is the position at the start of movement with the interactive function. This predetermined position (initial position) is a position set in advance in the system of the fourth embodiment, for example, position 3a.

[0391] [Interactive control of floating distance of floating images] Fig. 42 is an explanatory diagram of control based on Fig. 40, in which the floating distance of the space-floating image 3 is interactively changed in synchronization with the action of the operating finger 231 of the user 230 approaching the space-floating image 3, thereby changing the position of the imaging plane. As a prerequisite, the actuator activation switch 4101 is in an on state. Note that Fig. 42 etc. illustrates the position of the liquid crystal display panel 11 as a representative of the display unit (image display device 1).

[0392] In this case, the space-floating image display device 1000 captures the position of the viewpoint 232 (referred to as viewpoint position E1) of the user 230 who is in front of the space-floating image display device 1000 (for example, position Y1 on the Y axis in FIG. 40) using the camera of the imaging unit 1180. The viewpoint position E1 has coordinates (x1, y1, z1). At this time, the space-floating image display device 1000 estimates the distance (viewpoint distance) d of the viewpoint position E1 from the origin O (x-axis).

[0393] The appropriate coordinate range for the viewpoint position E1 of the user 230 (in other words, the range 232a where the floating image 3 in space can be suitably viewed, the eye box) is (x, y, z). The error tolerances corresponding to the axes (x, y, z), which are input and set in advance, are δx, δy, δz. Depending on the error tolerances, the appropriate coordinate range (x, y, z) for the viewpoint position E1 can be specified as -δx≦x≦δx, -δy≦y≦δy, (d-δz)≦z≦(d+δz).

[0394] The space floating image display device 1000 first captures the viewpoint position E1 with the camera as described above as the first control, and if the viewpoint position E1 is within the appropriate coordinate range, which is the allowable range, executes the subsequent second control. The first control is a processing procedure for confirming whether the user 230 is attempting to perform an aerial operation from an appropriate viewpoint. Note that the first control is not essential and may be omitted.

[0395] When the viewpoint position E1 is within the range 232a, which is the appropriate coordinate range, the space-floating image display device 1000 detects the approach of the operating finger 231 to the space-floating image 3 (particularly the movable range) using the camera of the imaging unit 1180. That is, the space-floating image display device 1000 detects the action of the user 230 moving their arm to bring the operating finger 231 closer to the space-floating image 3. In this embodiment, this approach is detected at a position on the z-axis closer to the user 230 than the frontmost position 3b (distance d2 where the floating distance is the longest) in the movable range of the space-floating image 3. The distance when the approach of the operating finger 231 is detected is set to distance df (df>d2), and the position is set to position 4201. The detected distance df (in other words, the distance threshold) may be a preset value.

[0396] Furthermore, in the initial state (i.e., when the operating finger 231 is not approaching), the space-floating image display device 1000 sets the display unit to position 1a (initial position), the space-floating image 3 to position 3a (initial position) at a distance d1, and the mid-air operation detection sensor 1351 to position 1351a (initial position) corresponding to position 3a. In FIG. 42 etc., the floating distance on the z-axis from the origin O corresponding to the x-axis (in other words, the protrusion distance) is indicated by d1, d2, and d3. Distance d1 is the distance corresponding to position 3a, the farthest back on the z-axis, of the space-floating image 3. Distance d2 is the distance corresponding to position 3b, the foremost front on the z-axis, of the space-floating image 3.

[0397] State A in Figure 43 shows the state when the approach of the operating finger 231 is detected. The tip of the operating finger 231 (231f) has reached position 4201 at a distance (approach distance) df. This approach state is detected based on the camera. Then, from the time of this approach detection, the space-floating image display device 1000 causes the display unit actuator 700 to gradually (for example, at a constant speed) move the display unit (image display device 1) in the -y direction (direction away from the x-axis) from the initial position 1a (11a) to position 1b (11b) according to the approach distance df. In particular, the liquid crystal display panel 11 moves from position 11a to position 11b.

[0398] As the display unit moves, the floating distance of the floating image 3 increases from the initial position 3a to position 3b. The floating distance gradually increases from distance d1. In addition to controlling the movement of the display unit, the floating image display device 1000 also controls the sensor actuator 800 to move the aerial operation detection sensor 1351 so that the detection surface follows the position of the floating image 3. In other words, the aerial operation detection sensor 1351 moves from position 1351a to position 1351b.

[0399] The space-floating image display device 1000 completes the movement of the display unit and the mid-air operation detection sensor 1351 at the exact timing (state B in FIG. 43) when it detects that the operating finger 231 has come into contact with the area of ​​the moving space-floating image 3 (for example, position 3c at distance d3). The space-floating image display device 1000 detects that the tip of the operating finger 231 has come into contact with the area of ​​the space-floating image 3 by the mid-air operation detection sensor 1351.

[0400] State B in Figure 43 shows a state when the operating finger 231 comes into contact with the area of ​​the floating image 3. Distance d3 is a distance corresponding to the position of the tip of the operating finger 231, which is between distance d1 and distance d2. Position 3c is a position corresponding to distance d3. Position 11c is a position on the liquid crystal display panel 11 corresponding to distance d3 (-d3).

[0401] The space-floating image display device 1000 sets the space-floating image 3 at position 3c at the time of contact, and stops the space-floating image 3. The space-floating image display device 1000 displays a predetermined screen (image content) on the space-floating image 3 stopped at position 3c. At this time, the mid-air operation detection sensor 1351 located at position 11c of the liquid crystal display panel 11 and position 1351c corresponding to position 3c of the space-floating image 3 can detect the mid-air operation on the screen of the space-floating image 3 at position 3c. After this, normal operation resumes. The user 230 can view the screen (image content) of the space-floating image 3 at position 3c, and perform mid-air operations (touch operations, etc.) on the screen with the operating finger 231.

[0402] 43 shows a case where the speed at which the operating finger 231 moves from the position at distance df to the back (-z) is roughly the same as the speed at which the imaging surface of the floating in space image 3 moves from the initial position 3a to the front (+z). The distance from the position at distance df to position 3c at distance d3 at the time of contact is roughly the same as the distance from position 3a at distance d1 to position 3c at distance d3 at the time of contact. The speed at which the floating in space image 3 is moved by the display unit may use, for example, a preset value.

[0403] The control of moving the imaging plane of the space-floating image 3 in the above interactive function is not limited to the above example. More generally, the floating distance of the space-floating image 3 may be changed at any speed (movement speed). Furthermore, the space-floating image display device 1000 may capture the speed of the operating finger 231 when it approaches with a camera, and determine the movement speed of the space-floating image 3 to match the captured speed.

[0404] Fig. 44 shows another example of the speed of the operating finger 231 and the speed of the floating-in-space image 3. State A is when the operating finger 231 moves from position 4201 at a distance df at a speed slower than the speed at which the floating-in-space image 3 moves. The position at the time of contact is position 3d, and the distance is distance d4. The position of the floating-in-space image 3 is at a position further forward on the z-axis than in states A and B of Fig. 43 (d4>d3).

[0405] State B is the case where the operating finger 231 moves from the position 4201 at a distance df at a speed faster than the speed of the movement of the floating-in-space image 3. The position at the time of contact is position 3e, and the distance is distance d5. The position of the floating-in-space image 3 is located further back on the z-axis than in states A and B of FIG. 43 (d5 <d3)。

[0406] 44 can be considered as a case where the speed of the operating finger 231 is kept constant and the speed of the movement of the floating image 3 in space is changed from large to small, in contrast to the example of FIG.

[0407] As in the above example, the position of the floating in space image 3 at the time of contact is determined according to the moving speed of the operating finger 231 or the moving speed of the floating in space image 3. Note that the stationary position of the floating in space image 3 may not necessarily be determined immediately upon contact detection, but may be determined after a certain period of time has elapsed after contact detection.

[0408] Furthermore, when controlling the movement of the above-mentioned space-floating image 3, the space-floating image display device 1000 displays any video content (for example, it may be a video showing the image moving) on ​​the space-floating image 3 while it is moving from its initial position. The user 230 can visually recognize the video content of the space-floating image 3 while it is moving. In other words, from the user 230's perspective, it feels as if the video content is moving towards the user in accordance with the operation finger 231, making it easy to recognize the interactive action. The video content displayed during the movement may be different from or the same as the video content displayed after the interactive position is set (after it stops).

[0409] 45 shows an example of a display image 5000 during interactive movement and a display image 5001 after the image has stopped. The display image 5000 is an example similar to the image during guidance in the third embodiment (FIG. 27A), and has an area 5000a that serves as a guide for directing the operating finger 231. The display image 5001 is an example similar to the image 3020 in FIG.

[0410] As described above, after the floating-in-space image 3 is interactively moved in response to the approach of the operating finger 231 and stopped upon contact, and then the floating-in-space image display device 1000 operates normally, the floating-in-space image 3 may be controlled to return to its initial state when a predetermined condition is met. The condition may be, for example, when the operating finger 231 moves forward on the z-axis with respect to the floating-in-space image 3 at position 3c at a distance d3 in state B of Fig. 43 and a certain amount of time has passed, or when the user 230 moves away from the floating-in-space image 3 and the front of the device and a certain amount of time has passed, or when a state in which no mid-air operation is detected continues for more than a predetermined time, etc.

[0411] When the above conditions are met, the space-floating image display device 1000 will temporarily erase the display of the space-floating image 3 and reset the position. That is, the space-floating image display device 1000 will return the space-floating image 3 to the initial position 3a by controlling the display unit actuator 700. After that, when the user 230 or the operating finger 231 approaches again, the space-floating image display device 1000 will start controlling the interactive function in the same way as described above.

[0412] As described above, in the fourth embodiment, when the operation finger 231 of the user 230 approaches the floating-in-space image 3, the floating-in-space image 3 is moved so as to approach the operation finger 231, and when the operation finger 231 and the floating-in-space image 3 come into contact with each other, the movement of the floating-in-space image 3 is stopped and the floating-in-space image 3 is set at that position. This gives the user 230 the impression that the floating-in-space image 3 is interactively responding to the movement of the operation finger 231, and that the position of the floating-in-space image 3 is set to follow the operation finger 231.

[0413] [Variations] The following modifications of the fourth embodiment are also possible. First, a configuration in which the housing actuator 600 is added to the configuration of Fig. 40, as in the third embodiment, is also possible. In this configuration, the tilt of the floating image 3 in space can be adjusted, and a function of moving and reacting the floating image 3 in space interactively can be realized.

[0414] Fig. 46 shows another modified example. First, similar to Fig. 42 etc., the space-floating image display device 1000 moves the space-floating image 3 in accordance with the approach of the operating finger 231, and temporarily sets the space-floating image 3 and the mid-air operation detection sensor 1351 to the position where they come into contact (for example, position 3c at a distance d3). Now, suppose that the user 230 moves the operating finger 231 slightly back and forth from the contact position 3c to a position that the user 230 feels is more suitable. The space-floating image display device 1000 allows this movement, and detects the position of the tip of the operating finger 231 after the movement by the camera or the mid-air operation detection sensor 1351. The space-floating image display device 1000 sets (i.e., fine-tunes) the space-floating image 3 to match the detected position after the movement.

[0415] In the illustrated example, the user 230 moves the operating finger 231 from position 3c at a distance d3 to position 3c' (floating distance is distance d3') a little further back than position 3c, and stops it there. The space-floating image display device 1000 moves the display unit etc. so as to set the space-floating image 3 at position 3d'. This gives the user 230 the feeling that by slightly pushing the space-floating image 3 at position 3c towards the back, the space-floating image 3 has been fine-tuned to position 3d.

[0416] The technology according to this embodiment displays high-resolution, high-brightness video information in a state where it floats in space, allowing users to operate the device without worrying about contact infection. Applying the technology according to this embodiment to a system used by an unspecified number of users reduces the risk of contact infection and makes it possible to provide a contactless user interface that can be used without anxiety. This contributes to the "Good Health and Well-Being" goal, one of the Sustainable Development Goals (SDGs) advocated by the United Nations.

[0417] Furthermore, the technology of this embodiment reduces the divergence angle of the emitted image light and aligns it with a specific polarization, thereby efficiently reflecting only the normal reflected light from the retroreflector, thereby enabling high light utilization efficiency and producing bright and clear floating images in space.The technology of this embodiment can provide a highly usable non-contact user interface that can significantly reduce power consumption.This contributes to the achievement of the United Nations' Sustainable Development Goals (SDGs): "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote technological innovation and infrastructure" and "Make cities and human settlements sustainable."

[0418] Although various embodiments have been described above in detail, the present invention is not limited to the above-described embodiments and includes various modifications. For example, the above-described embodiments are detailed descriptions of the entire system to clearly explain the present invention, and the present invention is not necessarily limited to a system including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations. [Explanation of symbols]

[0419] 1...image display device, 2...retroreflector, 3...space-floating image, 100...transparent member, 101...polarization separation member, 230...user, 231...operating finger, 232...viewpoint, 600...actuator for housing, 700...actuator for display unit, 800...actuator for sensor, 900...seat, 1000...space-floating image display device, 1180...imaging unit, 1190...housing, 1351...air operation detection sensor, 2001...direction of housing, 2002...emission direction.

Claims

1. A floating-in-the-air image display device, a video processing unit that performs video processing; a display unit that displays the image that has been processed by the image processing unit; an optical system that generates a floating image based on the image displayed by the display unit; a sensor for detecting a user's mid-air operation on the floating image; a display unit moving mechanism that moves the display unit along the output direction of the image; a sensor moving mechanism that moves the sensor; Equipped with By moving the display unit using the display unit movement mechanism, the position of the image of the floating-in-the-air image is moved along the emission direction of the image light that forms the floating-in-the-air image, and the sensor is moved by the sensor movement mechanism so as to align with the position of the image of the floating-in-the-air image. A floating video display device.

2. 2. The floating-in-the-air image display device according to claim 1, a housing that houses the display unit, the optical system, and the sensor; a housing movement mechanism that rotates and moves the housing; Equipped with The tilt of the image of the floating image is changed by rotating and moving the housing using the housing movement mechanism. A floating video display device.

3. 3. The airborne image display device according to claim 2, the housing movement mechanism has an actuator that rotates and moves the housing on an arc-shaped rail via wheels; A floating video display device.

4. 3. The airborne image display device according to claim 2, The housing movement mechanism has an actuator that rotates the housing around a rotation axis. A floating video display device.

5. 3. The airborne image display device according to claim 2, A floating image display device installed on a vehicle, With the user sitting in a seat of the vehicle, the positions of the display unit, the floating image, and the sensor are adjusted according to the user's viewpoint position and the state of the operating finger. A floating video display device.

6. 6. The airborne image display device according to claim 5, Equipped with an imaging unit, capturing a viewpoint position of the user based on the imaging unit; automatically controlling the rotational movement by the housing movement mechanism so that the emission direction of the video light of the floating video image is aligned with the viewpoint position of the user; A floating video display device.

7. 2. The floating-in-the-air image display device according to claim 1, setting the display unit to an initial position using the display unit movement mechanism, thereby setting the position of the image of the floating-in-the-air image to the initial position, and setting the initial position of the sensor using the sensor movement mechanism so as to align with the initial position of the floating-in-the-air image; a guide image is displayed on the floating-in-the-air image, thereby allowing the user to place an operation finger at a desired position on the floating-in-the-air image; moving the sensor from the initial position by the sensor moving mechanism, and detecting the position of the tip of the operating finger by the sensor; moving the display unit from the initial position by the display unit movement mechanism so as to align with the detected position of the tip of the operating finger, thereby moving the position of the image of the floating-in-the-air image from the initial position, and moving the sensor from the initial position by the sensor movement mechanism so as to align with the moved position of the image of the floating-in-the-air image; A floating video display device.

8. 8. The airborne image display device according to claim 7, the sensor is a sensor that emits and receives light parallel to an image plane of the floating image so that the image plane serves as a detection plane, moving the sensor from the initial position by the sensor moving mechanism, and obtaining information on whether the tip of the operating finger has been detected by the sensor; A floating video display device.

9. 8. The airborne image display device according to claim 7, the guide image displayed on the floating-in-the-air image has an image area serving as a guide for guiding the operation finger, and the image area serving as a guide is displayed at a position corresponding to an inclination of the floating-in-the-air image. A floating video display device.

10. 6. The airborne image display device according to claim 5, storing setting information regarding the setting of the position of the floating-in-the-air image for each user; the setting information includes information about the position of the floating-in-the-air image associated with a viewpoint position of the user when the user is sitting in a seat of the vehicle; Reproducing the position of the floating-in-the-air image based on the setting information in accordance with the user's viewpoint position; A floating video display device.

11. The airborne image display device according to claim 10, when the user's viewpoint position has changed from the previously set viewpoint position, calculate the amount of change in the viewpoint position, and adjust the position of the floating-in-the-air image according to the amount of change in the viewpoint position so that the relative positional relationship between the floating-in-the-air image and the viewpoint position is roughly maintained; A floating video display device.

12. 2. The floating-in-the-air image display device according to claim 1, Equipped with an imaging unit, capturing a viewpoint position of the user based on the imaging unit; calculating a visible area on the screen of the floating image in accordance with the user's viewpoint position; controlling the display magnification of the image on the display screen of the display unit so as to fit the image to the visible area; A floating video display device.

13. 2. The floating-in-the-air image display device according to claim 1, setting the display unit to an initial position using the display unit movement mechanism, thereby setting the position of the image of the floating-in-the-air image to the initial position, and setting the initial position of the sensor using the sensor movement mechanism so as to align with the initial position of the floating-in-the-air image; an initial position of the floating-in-the-air image and an initial position of the sensor are at positions on the far side of a movable range as viewed from the user; detecting the approach of the user's operating finger to the floating image; in response to the detection of the approach, the display unit moving mechanism moves the display unit from the initial position, thereby moving the floating-in-the-air image in a direction approaching the operating finger, and the sensor moving mechanism moves the sensor to match the position where the floating-in-the-air image is formed; detecting contact between the floating image and the operating finger by the sensor; freezing the floating image in accordance with the position of the detected contact; A floating video display device.

14. 14. The airborne image display device according to claim 13, Equipped with an imaging unit, detecting the approach of the operating finger based on the imaging unit; A floating video display device.

Images