Aerial image display device, product vending machine, and display device

Abstract

The present invention provides a more suitable aerial image display device. The present invention contributes to the Sustainable Development Goals (SDGs) of "3. Good health and well-being", "9. Industry, innovation and infrastructure" and "11. Sustainable cities and communities". An aerial image display device according to the present invention which is mounted on a product vending machine or a ticket vending machine constitutes an operation unit for operating the product vending machine or the ticket vending machine. The operation unit displays an aerial image for operating the product vending machine or the ticket vending machine, and, on the basis of detection of an operation with respect to the aerial image, executes a prescribed process by the product vending machine or the ticket vending machine. The prescribed process includes a process for selecting one of a plurality of candidates. The aerial image includes an object image for receiving an operation for selection.

Description

Floating image display device, product vending machine, and display device

[0001] The present invention relates to a floating-in-the-air image display device.

[0002] The floating information display technology is disclosed in, for example, Patent Document 1.

[0003] Japanese Patent Application Laid-Open No. 2019-128722

[0004] However, the disclosure of Patent Document 1 does not sufficiently consider configurations for obtaining practical brightness and quality for the floating image, or configurations for allowing users to view the floating image more enjoyably.

[0005] An object of the present invention is to provide a more suitable floating-in-the-air image display device.

[0006] To solve the above problem, for example, the configuration described in the claims is adopted. The present application includes multiple means for solving the above problem, and one example thereof may be configured as follows: A floating-in-the-air image display device mounted on a product vending machine or automatic ticket vending machine, the floating-in-the-air image display device comprising: an image processing unit that processes image; a display unit that displays image processed by the image processing unit; an optical system that generates a floating-in-the-air image based on the image displayed by the display unit; and a user operation detection mechanism that detects user operations on the display area of ​​the floating-in-the-air image, the floating-in-the-air image display device constituting an operation unit for operating the product vending machine or the automatic ticket vending machine, the operation unit displaying the floating-in-the-air image for operating the product vending machine or the automatic ticket vending machine, the operation unit causing the product vending machine or the automatic ticket vending machine to execute a predetermined process based on the detection of the operation on the floating-in-the-air image, the predetermined process including a process of selecting one from a plurality of candidates, and the floating-in-the-air image including an object image that accepts the operation for the selection.

[0007] According to the present invention, a more suitable floating-in-the-air image display device can be realized. Other problems, configurations, and effects will become clear in the following description of the embodiments.

[0008] FIG. 1 is a diagram showing an example of a usage form of a space-floating image display device according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of a configuration of a main part and a retroreflector of a space-floating image display device according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of a configuration of a main part and a retroreflector of a space-floating image display device according to an embodiment of the present invention. FIG. 4 is a diagram showing an example of a configuration of a main part and a retroreflector of a space-floating image display device according to an embodiment of the present invention. FIG. 5 is a diagram showing an example of a configuration of a main part and a retroreflector of a space-floating image display device according to an embodiment of the present invention. FIG. 6 is a projection diagram of a retroreflector constituting a space-floating image display device according to an embodiment of the present invention. FIG. 7 is a top view of a retroreflector constituting a space-floating image display device according to an embodiment of the present invention. FIG. 8 is a perspective view of a corner reflector constituting a retroreflector constituting a space-floating image display device according to an embodiment of the present invention. FIG. 9 is a top view of a corner reflector constituting a retroreflector constituting a space-floating image display device according to an embodiment of the present invention. FIG. 10 is a side view of a corner reflector constituting a retroreflector constituting a space-floating image display device according to an embodiment of the present invention. FIG. 11 is a diagram showing an example of a configuration of a space-floating image display device according to an embodiment of the present invention. FIG. 1 is a diagram showing an example of the configuration of a space-floating image display device according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of the configuration of a space-floating image display device according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of the configuration of a space-floating image display device according to an embodiment of the present invention. FIG. 4 is a diagram showing an example of the configuration of a space-floating image display device according to an embodiment of the present invention. FIG. 5 is a diagram showing an example of the configuration of a space-floating image display device according to an embodiment of the present invention. FIG. 6 is a diagram showing an example of the configuration of a space-floating image display device according to an embodiment of the present invention. FIG. 7 is a diagram showing an example of the configuration of a space-floating image display device according to an embodiment of the present invention. FIG. 8 is a diagram showing an example of the configuration of a space-floating image display device according to an embodiment of the present invention.FIG. 1 is a diagram showing an example of the configuration of a space-floating image display device according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of the configuration of a space-floating image display device according to an embodiment of the present invention. FIG. 3 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. FIG. 4 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. FIG. 5 is a cross-sectional view showing an example of a specific configuration of a light source device according to an embodiment of the present invention. FIG. 6 is a layout diagram showing main parts of a space-floating image display device according to an embodiment of the present invention. FIG. 7 is a cross-sectional view showing the configuration of a display device according to an embodiment of the present invention. FIG. 8 is a cross-sectional view showing the configuration of a display device according to an embodiment of the present invention. FIG. 9 is an explanatory diagram for explaining light source diffusion characteristics of an image display device according to an embodiment of the present invention. FIG. 10 is an explanatory diagram for explaining diffusion characteristics of an image display device according to an embodiment of the present invention. FIG. 11 is an explanatory diagram for explaining an example of a problem solved by image processing according to an embodiment of the present invention. FIG. 12 is an explanatory diagram for explaining an example of image processing according to an embodiment of the present invention. FIG. 13 is an explanatory diagram for explaining an example of image display processing according to an embodiment of the present invention. FIG. 14 is a diagram showing an example of the configuration of a main part and a retro-reflection part of a space-floating image display device according to an embodiment of the present invention. FIG. 15 is a diagram showing an example of the configuration of a product vending machine equipped with a space-floating image display device according to an embodiment. FIG. 16 is a diagram showing an example of the functional block configuration of a product vending machine according to an embodiment. FIG. 1 is a diagram showing an example of the configuration of a product vending machine function unit of a product vending machine according to an embodiment; FIG. 2 is a side view showing an example of mounting a space floating image display device on the housing of a product vending machine according to an embodiment; FIG. 3 is a side view showing an example of the configuration of a product display unit on the housing of a product vending machine according to an embodiment; FIG. 4 is a side view showing an example of the configuration of an operation unit on the housing of a product vending machine according to an embodiment; FIG. 5 is a side view showing an example of the configuration of an operation unit on the housing of a product vending machine according to an embodiment; FIG. 6 is a side view showing an example of the configuration of an operation unit on the housing of a product vending machine according to an embodiment; FIG. 7 is a side view showing an example of the configuration of an image of the operation unit according to an embodiment; FIG. 8 is a diagram showing a sequence between a product vending machine and a user according to an embodiment; FIG. 9 is an explanatory diagram related to detection of an aerial operation with respect to an image of the operation unit according to an embodiment; FIG. 10 is a side view showing an example of the configuration of a sensor for detecting an aerial operation according to an embodiment;1 is a side view showing an example of the configuration of a sensor for detecting an aerial operation according to an embodiment; FIG. 2 is a diagram showing an example of the configuration of an aerial operation detection sensor according to an embodiment; FIG. 3 is an explanatory diagram showing an example of image display control of an operation unit according to an embodiment; FIG. 4 is an explanatory diagram showing an example of image display control of an operation unit according to an embodiment; FIG. 5 is an explanatory diagram showing an example of the initial state of a product display unit according to an embodiment; FIG. 6 is an explanatory diagram showing an example of path control in a product display unit according to an embodiment; FIG. 7 is an explanatory diagram showing an example of path control in a product display unit according to an embodiment; FIG. 8 is a diagram showing an example of the configuration of cooperation between a product vending machine and a space-floating image display device according to an embodiment; FIG. 9 is a diagram showing an example of the configuration of a product vending machine with a partition provided in the operation unit according to an embodiment; FIG. 10 is a diagram showing an example of the configuration of a product vending machine with a partition provided in the operation unit according to an embodiment; FIG. 11 is a diagram showing an example of the configuration of a product vending machine with a partition provided in the operation unit according to an embodiment; FIG. 12 is a diagram showing an example of the configuration of a product vending machine with two openings provided in the operation unit according to an embodiment; FIG. 13 is a diagram showing an example of the configuration of a product vending machine with a partition provided in the operation unit according to an embodiment; FIG. 1 is a diagram showing an example of the configuration of the partition of the operation unit and the floating image in the depth direction according to an embodiment. FIG. 2 is a diagram showing an example of the configuration of the partition of the operation unit and the floating image in the depth direction according to an embodiment. FIG. 3 is a diagram showing an example of the configuration of the partition of the operation unit and the floating image in the depth direction according to an embodiment. FIG. 4 is a diagram showing an example of mid-air operation in the depth direction relative to the image on the operation unit according to an embodiment. FIG. 5 is a diagram showing an example of the configuration of the floating image on the operation unit according to an embodiment. FIG. 6 is a diagram showing an example of the configuration of a ticket vending machine equipped with a floating image display device according to an embodiment. FIG. 7 is a diagram showing an example of the configuration of a ticket vending machine function unit of a ticket vending machine according to an embodiment. FIG. 8 is a diagram showing an example of display control in a ticket vending machine according to an embodiment. FIG. 9 is a diagram showing an example of display control in a ticket vending machine according to an embodiment. FIG. 10 is a diagram showing an example of the screen configuration on the display of a ticket vending machine according to an embodiment. FIG. 11 is a diagram showing an example of the configuration of a partition on the operation unit of a ticket vending machine according to an embodiment. A diagram showing an example of the configuration of a ticket vending machine equipped with a space floating image display device according to an embodiment. A diagram showing an example of the configuration of a ticket vending machine display with a partition according to an embodiment. A diagram showing an example of the configuration of a ticket vending machine display with a partition according to an embodiment. A diagram showing an example of the configuration of a ticket vending machine display with a partition according to an embodiment. A side view showing an example of the configuration of a ticket vending machine display with a partition according to an embodiment.

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the description of the embodiments, and various changes and modifications can be made by those skilled in the art within the scope of the technical ideas disclosed in this specification. Furthermore, in all drawings used to explain the present invention, components having the same functions are given the same reference numerals, and repeated explanations thereof may be omitted.

[0010] The following examples relate to an image display device that can transmit an image generated by image light from an image light source through a transparent member that separates a space, such as glass, and display the image as a floating image outside the transparent member. In the following explanation of the examples, the image floating in space is expressed using the term "floating image in space." Instead of this term, it is also acceptable to express it as "aerial image," "spatial image," "floating image in space," "floating optical image of displayed image," "floating optical image of displayed image," etc. The term "floating image in space," which is mainly used in the explanation of the examples, is used as a representative example of these terms.

[0011] According to the following embodiments, a suitable image display device can be realized for, for example, bank ATMs, train station ticket machines, digital signage, and the like. For example, currently, bank ATMs, train station ticket machines, and the like typically use touch panels, but by using a transparent glass surface or a light-transmitting plate, high-resolution image information can be displayed in a floating state on the glass surface or light-transmitting plate. In this case, by making the divergence angle of the emitted image light small, i.e., an acute angle, and further aligning it with a specific polarization, only the normal reflected light is efficiently reflected by the retroreflector, resulting in high light utilization efficiency and suppressing the ghost images that occur in addition to the main floating image, which is a problem with conventional retroreflection systems, thereby achieving a clear floating image. Furthermore, a device including the light source of this embodiment can provide a novel, highly usable floating image display device (floating image display system) that can significantly reduce power consumption. Furthermore, a floating image display device for a vehicle can be provided that can display a so-called unidirectional floating image that can be viewed inside and / or outside the vehicle.

[0012] First Embodiment An example of the configuration of a space floating image display device will be described below as a first embodiment of the present invention.

[0013] <Example of Usage of the Space-Floating Image Display Device> Figure 1 is a diagram showing an example of usage of a space-floating image display device according to an embodiment of the present invention, and is a diagram showing the overall configuration of the space-floating image display device according to this embodiment. The specific configuration of the space-floating image display device will be described in detail using Figure 2 and other figures. Light with a narrow-angle directional characteristic and specific polarization is emitted from the image display device 1 as an image light beam, reflected by the optical system within the space-floating image display device, and then incident on the retroreflector 2, retroreflected and transmitted through a transparent member 100 (glass, etc.), forming a real aerial image (space-floating image 3) on the outside of the glass surface. In the following examples, the retroreflector 2 (retroreflector) will be used as an example of the retroreflector. However, the retroreflector 2 of the present invention is not limited to a planar plate, and is used as an example of a concept including a sheet-like retroreflector attached to a planar or non-planar member, or an entire assembly in which a sheet-like retroreflector is attached to a planar or non-planar member. Furthermore, since the light rays reflected by the retroreflector 2 have the optical property of forming an image, the retroreflector 2 may also be expressed as an imaging optical member or an imaging optical plate.

[0014] In addition, in a store or the like, a space is partitioned by a show window (also called "window glass") 105, which is a transparent member such as glass. According to the space floating image display device of this embodiment, it is possible to transmit such a transparent member and display a floating image in one direction to the outside and / or inside of the store (space).

[0015] 1, the inside of the window glass 105 (inside the store) is shown in the depth direction, and the outside (for example, the sidewalk) is shown in the foreground. On the other hand, by providing a means for reflecting specific polarized waves in the window glass 105, it is possible to form an aerial image at a desired position inside the store by reflecting the specific polarized waves.

[0016] <Configuration Example of Optical System of Space-Floating Image Display Device> Figure 2A is a diagram showing an example of the configuration of an optical system of a space-floating image display device according to one embodiment of the present invention. The configuration of the space-floating image display device will be described in more detail using Figure 2A. As shown in Figure 2A (1), a display device 1 that diverges specific polarized image light at a narrow angle is provided in an oblique direction of a transparent member 100 such as glass. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates specific polarized light with narrow-angle diffusion characteristics.

[0017] Image light of a specific polarization from the display device 1 is reflected by a polarization separation member 101 (in the figure, the polarization separation member 101 is formed in a sheet shape and adhered to the transparent member 100) provided on a transparent member 100 and has a film that selectively reflects image light of a specific polarization, and then enters the retroreflector 2. A λ / 4 plate 21 is provided on the image light incident surface of the retroreflector 2. The image light is polarized and converted from the specific polarization to the other polarization by passing through the λ / 4 plate 21 twice, once when entering the retroreflector 2 and once when exiting. Here, the polarization separation member 101, which selectively reflects image light of a specific polarization, has the property of transmitting the polarized light of the other polarization that has been polarization-converted, so the image light of the specific polarization after polarization conversion passes through the polarization separation member 101. The image light that has passed through the polarization separation member 101 forms a spatially floating image 3, which is a real image, outside the transparent member 100. 2A shows an example in which the chief ray of the image light incident on the retroreflector 2 is incident at an angle of 90° to the retroreflector 2. However, the incident angle of the chief ray of the image light on the retroreflector 2 is not limited to 90°, and an angle of, for example, 90°±15° can also be used.

[0018] Here, a first example of polarization design for the optical system of FIG. 2A will be described. For example, a configuration may be adopted in which S-polarized (S stands for Senkrecht; polarized light whose electric field oscillates perpendicular to the plane of incidence) image light is emitted from the display device 1 to the polarization separation member 101, and the polarization separation member 101 has the property of reflecting S-polarized light and transmitting P-polarized (P stands for parallel; polarized light whose electric field oscillates within the plane of incidence) light. In this case, the S-polarized image light that reaches the polarization separation member 101 from the display device 1 is reflected by the polarization separation member 101 and proceeds toward the retroreflector 2. When the image light is reflected by the retroreflector 2, it passes twice through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2, and the image light is converted from S-polarized to P-polarized light. The P-polarized image light then proceeds toward the polarization separation member 101 again. Here, the polarization separation member 101 has the property of reflecting S-polarized light and transmitting P-polarized light, so the P-polarized image light passes through the polarization separation member 101 and then through the transparent member 100. The image light that passes through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a space-floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101. This polarization design makes it possible to appropriately form the space-floating image 3.

[0019] Next, a second example of polarization design for the optical system of FIG. 2A will be described. For example, a configuration may be adopted in which P-polarized image light is emitted from the display device 1 to the polarization separation member 101, and the polarization separation member 101 has the property of reflecting P-polarized light and transmitting S-polarized light. In this case, the P-polarized image light that reaches the polarization separation member 101 from the display device 1 is reflected by the polarization separation member 101 and proceeds toward the retroreflector 2. When the image light is reflected by the retroreflector 2, it passes twice through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2, thereby converting the image light from P-polarized light to S-polarized light. The image light converted to S-polarized light proceeds again toward the polarization separation member 101. Here, the polarization separation member 101 has the property of reflecting P-polarized light and transmitting S-polarized light, so the S-polarized image light passes through the polarization separation member 101 and then through the transparent member 100. The image light transmitted through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101. This polarization design allows the floating image 3 to be formed optimally.

[0020] The light forming the space-floating image 3 is a collection of light rays that converge from the retroreflector 2 to the optical image of the space-floating image 3, and these light rays continue to travel in a straight line even after passing through the optical image of the space-floating image 3. Therefore, the space-floating image 3 is an image with high directionality, unlike the diffuse image light formed on a screen by a general projector or the like. Therefore, in the configuration of FIG. 2A, when a user views the space-floating image 3 from the direction of arrow A, the space-floating image 3 is perceived as a bright image. However, when another person views the space-floating image 3 from the direction of arrow B, the space-floating image 3 cannot be seen as an image at all. This characteristic is very suitable for use in systems that display images that require high security or highly confidential images that should be kept secret from people directly facing the user.

[0021] Depending on the performance of the retroreflector 2, the polarization axis of the reflected image light may become irregular. The reflection angle may also become irregular. Such irregular light may not maintain the polarization state and propagation angle assumed in the design. For example, light with an unintended polarization state and propagation angle may re-enter the image display surface of the liquid crystal display panel 11 directly from the position of the retroreflector 2 without passing through the polarization separation member. Such light with an unintended polarization state and propagation angle may be reflected by components within the space-floating image display device and then re-enter the image display surface of the liquid crystal display panel 11. Such light re-entering the image display surface of the liquid crystal display panel 11 may be re-reflected by the image display surface of the liquid crystal display panel 11 constituting the display device 1, potentially generating ghost images and degrading the image quality of the space-floating image. Therefore, in this embodiment, an absorbing polarizer 12 may be provided on the image display surface of the display device 1. The image light emitted from the display device 1 is transmitted through the absorbing polarizer 12, and the reflected light returning from the polarization separation member 101 is absorbed by the absorbing polarizer 12, thereby suppressing the re-reflection. This makes it possible to prevent degradation of image quality due to ghost images of spatially floating images. Specifically, if the display device 1 is configured to emit S-polarized image light to the polarization separation member 101, the absorbing polarizer 12 may be a polarizer that absorbs P-polarized light. Furthermore, if the display device 1 is configured to emit P-polarized image light to the polarization separation member 101, the absorbing polarizer 12 may be a polarizer that absorbs S-polarized light.

[0022] The polarization separation member 101 may be formed of, for example, a reflective polarizing plate or a metal multilayer film that reflects specific polarized waves.

[0023] 2A(2) shows an example of the surface shape of a typical retroreflector 2. A prism body with a regularly arranged array of triangular pyramidal recessed reflective surfaces is arranged on the retroreflector 2. Light rays incident on the array of triangular pyramidal recessed surfaces are reflected by the multiple reflective surfaces of the triangular pyramidal recessed surfaces and are emitted as retroreflected light in a direction corresponding to the incident light, thereby displaying a real image floating in space based on the image displayed on the display device 1.

[0024] The resolution of this floating image in space depends not only on the resolution of the liquid crystal display panel 11, but also on the outer diameter D and pitch P of the retroreflective portion of the retroreflector 2 shown in Figure 2A (2). For example, when using a 7-inch WUXGA (1920 x 1200 pixels) liquid crystal display panel, even if one pixel (one triplet) is approximately 80 μm, if the diameter D of the retroreflective portion is 240 μm and the pitch is 300 μm, one pixel of the floating image in space will be equivalent to 300 μm. As a result, the effective resolution of the floating image in space will be reduced to about one-third.

[0025] Therefore, in order to make the resolution of the spatial floating image equivalent to that of the display device 1, it is desirable to make the diameter and pitch of the retroreflective portion close to one pixel of the liquid crystal display panel. On the other hand, in order to suppress the occurrence of moire caused by the retroreflective plate and the pixels of the liquid crystal display panel, it is advisable to design the pitch ratio of each to be a different integer multiple of one pixel. Furthermore, it is advisable to arrange the shape so that none of the sides of the retroreflective portion overlaps any of the sides of one pixel of the liquid crystal display panel.

[0026] The surface shape of the retroreflector according to this embodiment is not limited to the above example. Various surface shapes that achieve retroreflection may be used. Specifically, the surface of the retroreflector according to this embodiment may be provided with retroreflection elements in which triangular pyramidal prisms, hexagonal pyramidal prisms, other polygonal prisms, multi-vertex prisms, or combinations thereof are periodically arranged. Alternatively, the surface of the retroreflector according to this embodiment may be provided with retroreflection elements in which these prisms are periodically arranged to form cube corners. These may also be referred to as corner reflector arrays or polyhedral reflector arrays. Alternatively, the surface of the retroreflector according to this embodiment may be provided with capsule lens-type retroreflection elements in which glass beads are periodically arranged. The detailed configuration of these retroreflection elements can be achieved using existing technology, so a detailed description will be omitted. Specifically, the techniques disclosed in Japanese Patent Laid-Open Nos. 2001-33609, 2001-264525, 2005-181555, 2008-70898, and 2009-229942 may be used.

[0027] <Another configuration example 1 of the optical system of the space-floating image display device> Another configuration example of the optical system of the space-floating image display device will be described using Fig. 2B. In Fig. 2B, components with the same reference numerals as Fig. 2A have the same functions and configurations as Fig. 2A. For such components, repeated explanations will be omitted to simplify the explanation.

[0028] In the optical system of FIG. 2B , similar to FIG. 2A , image light of a specific polarization is output from the display device 1. The image light of a specific polarization output from the display device 1 is input to a polarization separation member 101B. The polarization separation member 101B is a member that selectively transmits image light of a specific polarization. Unlike the polarization separation member 101 of FIG. 2A , the polarization separation member 101B is not integrated with the transparent member 100 but has an independent plate-like shape. Therefore, the polarization separation member 101B may also be referred to as a polarization separation plate. The polarization separation member 101B may be configured as a reflective polarizing plate configured by attaching a polarization separation sheet to a transparent member. Alternatively, the transparent member may be formed of a metal multilayer film that selectively transmits specific polarization and reflects polarization of other specific polarizations. In FIG. 2B , the polarization separation member 101B is configured to transmit image light of a specific polarization output from the display device 1.

[0029] The image light that has passed through the polarization separation member 101B is incident on the retroreflector 2. A λ / 4 plate 21 is provided on the image light incident surface of the retroreflector. The image light is polarized and converted from a specific polarization to the other polarization by passing through the λ / 4 plate 21 twice, once when it enters the retroreflector and once when it leaves. Here, the polarization separation member 101B has the property of reflecting the polarized light of the other polarization that has been polarized and converted by the λ / 4 plate 21, so the image light after polarization conversion is reflected by the polarization separation member 101B. The image light reflected by the polarization separation member 101B passes through the transparent member 100 and forms a space-floating image 3, which is a real image, outside the transparent member 100.

[0030] Here, a first example of polarization design for the optical system of FIG. 2B will be described. For example, a configuration may be adopted in which P-polarized image light is emitted from the display device 1 to the polarization separation member 101B, and the polarization separation member 101B has the property of reflecting S-polarized light and transmitting P-polarized light. In this case, the P-polarized image light that reaches the polarization separation member 101B from the display device 1 passes through the polarization separation member 101B and proceeds to the retroreflector 2. When the image light is reflected by the retroreflector 2, it passes twice through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2, and the image light is converted from P-polarized light to S-polarized light. The image light converted to S-polarized light proceeds again to the polarization separation member 101B. Here, the polarization separation member 101B has the property of reflecting S-polarized light and transmitting P-polarized light, so the S-polarized image light is reflected by the polarization separation member 101 and passes through the transparent member 100. The image light transmitted through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101B. This polarization design allows the floating image 3 to be formed optimally.

[0031] Next, a second example of polarization design for the optical system of FIG. 2B will be described. For example, S-polarized image light may be emitted from the display device 1 to the polarization separator 101B, which may have the property of reflecting P-polarized light and transmitting S-polarized light. In this case, the S-polarized image light reaching the polarization separator 101B from the display device 1 passes through the polarization separator 101B and proceeds to the retroreflector 2. When the image light is reflected by the retroreflector 2, it passes twice through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2, thereby converting the S-polarized image light to P-polarized light. The P-polarized image light then proceeds again to the polarization separator 101B. Here, the polarization separator 101B has the property of reflecting P-polarized light and transmitting S-polarized light, so the P-polarized image light is reflected by the polarization separator 101 and passes through the transparent member 100. The image light transmitted through the transparent member 100 is light generated by the retroreflector 2, and therefore forms a floating image 3, which is an optical image of the image displayed on the display device 1, at a position that is in a mirror relationship with the image displayed on the display device 1 relative to the polarization separation member 101B. This polarization design allows the floating image 3 to be formed optimally.

[0032] In FIG. 2B , the image display surface of the display device 1 and the surface of the retroreflector 2 are arranged parallel to each other. The polarization separation member 101B is arranged at an angle α (e.g., 30°) with respect to the image display surface of the display device 1 and the surface of the retroreflector 2. Then, upon reflection by the polarization separation member 101B, the direction of travel of the image light reflected by the polarization separation member 101B (the direction of the chief ray of the image light) differs by an angle β (e.g., 60°) from the direction of travel of the image light incident from the retroreflector 2 (the direction of the chief ray of the image light). With this configuration, in the optical system of FIG. 2B , the image light is output at a predetermined angle shown toward the outside of the transparent member 100, forming a real image, the floating-in-space image 3. In the configuration of FIG. 2B , when viewed by a user from the direction of arrow A, the floating-in-space image 3 is perceived as a bright image. However, when viewed by another person from the direction of arrow B, the floating-in-space image 3 cannot be perceived as an image at all. This characteristic is extremely suitable for use in a system that displays images that require high security or highly confidential images that should be concealed from people directly facing the user.

[0033] As described above, the optical system of FIG. 2B has a different configuration from the optical system of FIG. 2A, but can form a suitable floating image in space, similar to the optical system of FIG. 2A.

[0034] An absorptive polarizing plate may be provided on the surface of the transparent member 100 facing the polarization separation member 101B. This absorptive polarizing plate may transmit the polarized light of the image light from the polarization separation member 101B and absorb the polarized light that is 90° out of phase with the polarized light of the image light from the polarization separation member 101B. In this way, the image light for forming the space-floating image 3 can be sufficiently transmitted while reducing the external light incident on the space-floating image 3 side of the transparent member 100 by approximately 50%. This makes it possible to reduce stray light in the optical system of FIG. 2B due to the external light incident on the space-floating image 3 side of the transparent member 100.

[0035] <Another configuration example 2 of the optical system of the space-floating image display device> Another configuration example of the optical system of the space-floating image display device will be described using Fig. 2C. Note that in Fig. 2C, components with the same reference numerals as Fig. 2B have the same functions and configurations as Fig. 2B. For the sake of simplicity, such components will not be described repeatedly.

[0036] The only difference between the optical system of Fig. 2B and the optical system of Fig. 2C is the arrangement angle of the polarization separation member 101B with respect to the image display surface of the display device 1 and the surface of the retroreflector 2. All other configurations are the same as those of the optical system of Fig. 2B, so repeated explanations will be omitted. The polarization design of the optical system of Fig. 2C is also the same as that of the optical system of Fig. 2B, so repeated explanations will be omitted.

[0037] In the optical system of FIG. 2C , the polarization separation member 101B is tilted at an angle α with respect to the image display surface of the display device 1 and the surface of the retroreflector 2. In FIG. 2C , the angle α is 45°. With this configuration, when the polarization separation member 101B reflects, the angle β between the direction of propagation of the image light incident from the retroreflector 2 (the direction of the chief ray of the image light) and the direction of propagation of the image light reflected by the polarization separation member 101B (the direction of the chief ray of the image light) is 90°. With this configuration, the image display surface of the display device 1 and the surface of the retroreflector 2 are perpendicular to the direction of propagation of the image light reflected by the polarization separation member 101B, thereby simplifying the angular relationship between the surfaces that make up the optical system. If the surface of the transparent member 100 is positioned perpendicular to the direction of propagation of the image light reflected by the polarization separation member 101B, the angular relationship between the surfaces that make up the optical system can be further simplified. In the configuration of Figure 2C, when a user views the floating image 3 from the direction of arrow A, the floating image 3 is perceived as a bright image. However, when another person views the floating image 3 from the direction of arrow B, the floating image 3 cannot be seen as an image at all. This characteristic is very suitable for use in a system that displays images that require high security or highly confidential images that should be kept secret from people directly facing the user.

[0038] As described above, the optical system of Fig. 2C has a different configuration from the optical systems of Fig. 2A and Fig. 2B, but can form a suitable floating image in space similar to the optical systems of Fig. 2A and Fig. 2B. In addition, the angles of the surfaces constituting the optical system can be made simpler.

[0039] An absorptive polarizing plate may be provided on the surface of the transparent member 100 facing the polarization separation member 101B. This absorptive polarizing plate may transmit the polarized light of the image light from the polarization separation member 101B and absorb the polarized light that is 90° out of phase with the polarized light of the image light from the polarization separation member 101B. In this way, the image light for forming the space-floating image 3 can be sufficiently transmitted while reducing the external light incident on the space-floating image 3 side of the transparent member 100 by approximately 50%. This makes it possible to reduce stray light in the optical system of FIG. 2C due to the external light incident on the space-floating image 3 side of the transparent member 100.

[0040] <Another Configuration Example 3 of the Optical System of the Space-Floating Image Display Device> Another configuration example of the optical system of the space-floating image display device will be described using FIG. 2D. The optical system of FIG. 2D is an optical system that uses a retroreflector 5 that is different from the retroreflector 2 used in FIGS. 2A to 2C. Hereinafter, Another Configuration Example 3 of the optical system will be described in more detail using FIGS. 2D to 2I. In FIG. 2D, components that are assigned the same reference numerals as those in FIGS. 2A to 2C have the same functions and configurations as those in FIGS. 2A to 2C. Such components will not be described repeatedly in order to simplify the explanation.

[0041] 2D is a diagram showing an example of the main components and retroreflection components of a space-floating image display device according to an embodiment of the present invention. A display device 1 that emits image light is provided obliquely on a transparent member 100 such as glass. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates light.

[0042] A chief ray 9020 representing the light beam emitted from the display device 1 travels toward the retroreflector 5 and is incident on the retroreflector 5 at an incident angle α. The incident angle α may be, for example, 45°. However, the incident angle α is not limited to 45°, and may also be, for example, 45°±15°.

[0043] The retroreflector 5 is an optical element having the optical property of retroreflecting light rays in at least some directions. Furthermore, since the reflected light rays have the optical property of forming an image, the retroreflector 5 may also be referred to as an imaging optical element or an imaging optical plate.

[0044] 2E, 2F, etc., the principal ray 9020 travels in the z direction and is retroreflected in the x and y directions by the retroreflector 5. As a result, the reflected light ray 9021 travels in a direction away from the retroreflector 5 along an optical path that is mirror-symmetrical with respect to the principal ray 9020 with the retroreflector 5 as the reference, passes through the transparent member 100, and forms the floating image 3 in space as a real image on the imaging plane.

[0045] The light beam forming the space-floating image 3 is a collection of light rays that converge from the retroreflector 5 to the optical image of the space-floating image 3, and these light rays continue to travel in a straight line even after passing through the optical image of the space-floating image 3. Therefore, the space-floating image 3 is an image with high directionality, unlike a diffuse image formed on a screen by a general projector or the like. Therefore, in the configuration of FIG. 2, when a user views the space-floating image 3 from the direction of arrow A, the space-floating image 3 is perceived as a bright image. However, when another person views the space-floating image 3 from the direction of arrow B, the space-floating image 3 cannot be perceived as an image at all. This characteristic is suitable for use in a system that displays images that require high security or highly confidential images that should be kept secret from people directly facing the user.

[0046] An example of the configuration of the retroreflector 5 will be described using Figures 2E and 2F. The retroreflector 5 has a configuration in which multiple corner reflectors 9040 are arranged in an array on the surface of a transparent member. This may also be called a corner reflector array or a polyhedral reflector array. The specific configuration of the corner reflector 9040 will be described in detail using Figures 2G, 2H, and 2I. Light rays 9111, 9112, 9113, and 9114 emitted from a light source 9110 are reflected twice by two mirror surfaces 9041 and 9042 of the corner reflector 9040, becoming reflected light rays 9121, 9122, 9123, and 9124. This double reflection is retroreflection in the x and y directions, where the light is reflected back in the same direction as the incident direction (traveling in a direction rotated 180 degrees), and in the z direction, where the angle of incidence and the angle of reflection match due to total reflection.

[0047] That is, the light rays 9111 to 9114 generate reflected light rays 9121 to 9124 on straight lines symmetrical in the z direction with respect to the corner reflector 9040, forming an aerial real image 9120. Note that the light rays 9111 to 9114 emitted from the light source 9110 are four light rays representing the diffused light from the light source 9110, and although the light rays incident on the retroreflector 5 are not limited to these, depending on the diffusion characteristics of the light source 9110, all incident light rays cause similar reflections and form an aerial real image 9120. Note that to make the drawing easier to see, the position of the light source 9110 and the position of the aerial real image 9120 are shown shifted in the x direction, but in reality, the position of the light source 9110 and the position of the aerial real image 9120 in the x direction are the same, and are overlapping when viewed from the z direction.

[0048] 2G, 2H, and 2I, the configuration and effects of the corner reflector 9040 that constitutes the retroreflector 5 will be described. The corner reflector 9040 is a rectangular parallelepiped with only two specific surfaces being mirror surfaces 9041 and 9042, and the other four surfaces being made of transparent materials. The retroreflector 5 has a configuration in which these corner reflectors 9040 are arrayed so that the corresponding mirror surfaces face in the same direction.

[0049] When viewed from the top (+z direction), a light ray 9111 emitted from the light source 9110 enters the mirror surface 9041 (or the mirror surface 9042) at a specific angle of incidence, is totally reflected at the reflection point 9130, and then is totally reflected again at the reflection point 9132 on the mirror surface 9042 (or the mirror surface 9041).

[0050] If the angle of incidence of light ray 9111 with respect to mirror surface 9041 (or mirror surface 9042) is θ, then the angle of incidence of first reflected light ray 9131 reflected by mirror surface 9041 (or mirror surface 9042) with respect to mirror surface 9042 (or mirror surface 9041) can be expressed as 90°-θ. Therefore, with respect to light ray 9111, second reflected light ray 9121 undergoes a rotation of 2θ after the first reflection and 2×(90°-θ) after the second reflection, resulting in a total reversal optical path of 180°. On the other hand, when viewed from the side (the direction halfway between -x and -y), total reflection in the z direction occurs only once. Therefore, if the angle of incidence with respect to mirror surface 9041 or mirror surface 9042 is φ, then reflected light ray 9121 undergoes a rotation of 2×φ after one reflection with respect to light ray 9111.

[0051] As described above, the light rays incident on the corner reflector 9040 undergo retroreflection, which results in an inverted optical path in the x and y directions, and specular reflection due to total reflection in the z direction. Considering the retroreflector 5, similar reflection occurs in each optical path, so that an image is formed at a point symmetrical with respect to the z axis direction by an inverted optical path that is convergent in the x and y directions.

[0052] 2A to 2C, the retroreflector 2 has retroreflection properties in three axes. As a result, when a diffusive incident light beam is incident on the retroreflector 2, a convergent reflected light beam travels toward the side of the retroreflector 2 where the light source of the incident light is located. The convergent reflected light beam forms an image in the air, forming a floating image 3. The traveling direction of the chief ray of the convergent reflected light beam reflected from the retroreflector 2 is opposite to the traveling direction of the chief ray of the diffusive incident light beam that is incident on the retroreflector 2.

[0053] 2D, the retroreflector 5 has retroreflection properties in two axial directions and specular reflection in the other axial direction. As a result, when a diffusive incident light beam is incident on the retroreflector 5, the convergent reflected light beam is reflected by the corner reflector array and travels in the direction opposite to the side of the retroreflector 5 where the light source of the incident light is located. The convergent reflected light beam forms an image in the air and forms the space floating image 3.

[0054] The traveling direction of the chief ray of the convergent reflected light beam reflected by the corner reflector array of the retroreflector 5 is not the opposite direction to the traveling direction of the chief ray of the diffusive incident light beam that is incident on the retroreflector 5. The normal direction component of the plate-shaped surface of the retroreflector 5 in the traveling direction of the chief ray of the diffusive incident light beam that is incident on the retroreflector 5 and the normal direction component of the plate-shaped surface of the retroreflector 5 in the traveling direction of the chief ray after being reflected by the retroreflector 5 and becoming a convergent reflected light beam, travel in a straight line, unchanged before and after reflection by the corner reflector array.

[0055] That is, the diffusive incident light beam is converted into a convergent reflected light beam by reflection on the retroreflector 5, but in the normal direction to the plate-shaped surface of the retroreflector 5, the light beam travels as if passing through the retroreflector 5. Here, the diffusive incident light beam incident on the retroreflector 5 and the convergent reflected light beam emerging from the retroreflector 5 are in a geometrically symmetrical relationship with respect to the plate-shaped surface of the retroreflector 5.

[0056] The resolution of the space-floating image formed by the light beams from the display device 1 depends not only on the resolution of the liquid crystal display panel 11, but also on the diameter D and pitch P (not shown) of the retroreflective portion of the retroreflector 5 shown in Figures 2E and 2F. For example, when using a 7-inch WUXGA (1920 x 1200 pixels) liquid crystal display panel, even if one pixel (one triplet) is approximately 80 μm, if the diameter D of the retroreflective portion is 240 μm and the pitch P is 300 μm, one pixel of the space-floating image will be equivalent to 300 μm. As a result, the effective resolution of the space-floating image will be reduced to about one-third.

[0057] Therefore, in order to make the resolution of the spatial floating image equivalent to that of the display device 1, it is desirable to make the diameter D and pitch P of the retroreflective portion close to one pixel of the liquid crystal display panel. On the other hand, in order to suppress the occurrence of moire due to the retroreflective plate and the pixels of the liquid crystal display panel, it is advisable to design the pitch ratio of each to be a different integer multiple of one pixel. Furthermore, it is advisable to arrange the shape so that none of the sides of the retroreflective portion overlaps any of the sides of one pixel of the liquid crystal display panel.

[0058] The shape of the retroreflector (imaging optical plate) according to this embodiment is not limited to the above example. It may have various shapes that achieve retroreflection. Specifically, it may be a variety of cubic corner bodies, corner reflector arrays, slit mirror arrays, dihedral corner reflector arrays, polyhedral reflector arrays, or shapes in which a combination of these reflective surfaces is periodically arranged. Alternatively, capsule lens-type retroreflecting elements in which glass beads are periodically arranged may be provided on the surface of the retroreflector according to this embodiment. Since existing technology can be used for the detailed configuration of these retroreflecting elements, a detailed description will be omitted. Specifically, the technology disclosed in JP 2017-33005 A, JP 2019-133110 A, JP 2017-67933 A, WO 2009 / 131128 A, etc. may be used.

[0059] 2D, the image light emitted from the display device 1 may be in any polarization state, either S-polarized or P-polarized.

[0060] As described above, the optical system of FIG. 2D is an optical system that uses a retroreflector different from the optical systems of FIGS. 2A to 2C, but it can form a more suitable floating image in space, similar to the optical systems of FIGS. 2A to 2C.

[0061] According to the optical systems of FIGS. 2A, 2B, 2C, and 2D described above, it is possible to provide brighter and higher quality floating images in space.

[0062] <<Block Diagram of Internal Configuration of Space-Floating Image Display Device>> Next, a block diagram of the internal configuration of the space-floating image display device 1000 will be described. Fig. 3 is a block diagram showing an example of the internal configuration of the space-floating image display device 1000.

[0063] The space-floating image display device 1000 includes a retroreflection unit 1101, an image display unit 1102, a light guide 1104, a light source 1105, a power supply 1106, an external power supply input interface 1111, an operation input unit 1107, a nonvolatile memory 1108, a memory 1109, a control unit 1110, a video signal input unit 1131, an audio signal input unit 1133, a communication unit 1132, an aerial operation detection sensor 1351, an aerial operation detection unit 1350, an audio output unit 1140, a microphone 1139, an image control unit 1160, a storage unit 1170, an imaging unit 1180, etc. In addition, the space-floating image display device 1000 may also include a removable media interface 1134, an attitude sensor 1113, a transmissive self-luminous image display device 1650, a second display device 1680, or a secondary battery 1112.

[0064] Each component of the space floating image display device 1000 is disposed in a housing 1190. Note that the imaging unit 1180 and the mid-air operation detection sensor 1351 shown in FIG.

[0065] The retroreflector 1101 in Figure 3 corresponds to the retroreflector 2 in Figures 2A, 2B, and 2C. The retroreflector 1101 retroreflects light modulated by the image display unit 1102. Of the light reflected from the retroreflector 1101, the light output to the outside of the space-floating image display device 1000 forms the space-floating image 3. When the optical system in Figure 2D is applied, the retroreflector 1101 corresponds to the retroreflector 5 in Figure 2D.

[0066] The image display unit 1102 in Fig. 3 corresponds to the liquid crystal display panel 11 in Fig. 2A, 2B, and 2C. The light source 1105 in Fig. 3 corresponds to the light source device 13 in Fig. 2A, 2B, and 2C. The image display unit 1102, the light guide 1104, and the light source 1105 in Fig. 3 correspond to the display device 1 in Fig. 2A, 2B, and 2C.

[0067] The video display unit 1102 is a display unit that generates a video by modulating transmitted light based on a video signal input under the control of a video control unit 1160 (described later). The video display unit 1102 (the aforementioned liquid crystal display panel 11) may be, for example, a transmissive liquid crystal panel, but is not limited to this. Alternatively, the video display unit 1102 may be, for example, a reflective liquid crystal panel that modulates reflected light, a DMD (Digital Micromirror Device: registered trademark) panel, or the like.

[0068] The light source 1105 generates light for the image display unit 1102 and is a solid-state light source such as an LED light source (LED: Light Emitting Diode) or a laser light source. The power supply 1106 converts AC current input from the outside via the external power supply input interface 1111 into DC current and supplies power to the light source 1105. The power supply 1106 also supplies the necessary DC current to each component within the space-floating image display device 1000. The secondary battery 1112 stores the power supplied from the power supply 1106. The secondary battery 1112 also supplies power to the light source 1105 and other components requiring power via the external power supply input interface 1111 when power is not supplied from the outside. In other words, when the space-floating image display device 1000 is equipped with the secondary battery 1112, the user can use the space-floating image display device 1000 even when power is not supplied from the outside.

[0069] The light guide 1104 guides light generated by the light source 1105 and irradiates it onto the video display unit 1102. The combination of the light guide 1104 and the light source 1105 can also be called a backlight for the video display unit 1102. The light guide 1104 may be configured mainly using glass. The light guide 1104 may be configured mainly using plastic. The light guide 1104 may be configured using a mirror. Various methods are possible for combining the light guide 1104 and the light source 1105. Specific configuration examples of the combination of the light guide 1104 and the light source 1105 will be described in detail later.

[0070] The aerial operation detection sensor 1351 is a sensor that detects an operation of the floating in space image 3 by an operating object such as a user's finger. The aerial operation detection sensor 1351 senses, for example, an area that overlaps with the entire display area of ​​the floating in space image 3. Note that the aerial operation detection sensor 1351 may only sense an area that overlaps with at least a portion of the display area of ​​the floating in space image 3.

[0071] Specific examples of the aerial operation detection sensor 1351 include a distance sensor that uses invisible light such as infrared light, an invisible light laser, ultrasonic waves, etc. The aerial operation detection sensor 1351 may also be configured to detect coordinates on a two-dimensional plane by combining multiple sensors. The aerial operation detection sensor 1351 may also be configured with a ToF (Time of Flight) LiDAR (Light Detection and Ranging) sensor or an image sensor.

[0072] The mid-air operation detection sensor 1351 only needs to be capable of sensing to detect touch operations with the user's finger on objects displayed as the floating-in-space image 3. Such sensing can be performed using existing technology.

[0073] The aerial operation detection unit 1350 acquires a sensing signal from the aerial operation detection sensor 1351 and, based on the sensing signal, determines whether or not the user's finger has touched an object in the floating-in-space image 3, and calculates the position (contact position) where the user's finger has touched the object. The aerial operation detection unit 1350 is configured, for example, by a circuit such as an FPGA (Field Programmable Gate Array). Some of the functions of the aerial operation detection unit 1350 may be realized by software, for example, by a spatial operation detection program executed by the control unit 1110 or the image control unit 1160. The aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be configured as an integrated unit. The aerial operation detection unit 1350 and the control unit 1110 or the image control unit 1160 may be configured as an integrated unit.

[0074] The aerial operation detection sensor 1351 and the aerial operation detection unit 1350 may be configured to be built into the space-floating image display device 1000, or may be provided externally as separate entities from the space-floating image display device 1000. When provided as separate entities from the space-floating image display device 1000, the aerial operation detection sensor 1351 and the aerial operation detection unit 1350 are configured to be able to transmit information and signals to the space-floating image display device 1000 via a wired or wireless communication connection path or a video signal transmission path. This makes it possible to build a system in which the space-floating image display device 1000, which does not have aerial operation detection function, is used as the main body, and only the aerial operation detection function can be added as an option.

[0075] Also, the aerial operation detection sensor 1351 may be a separate unit, and the aerial operation detection unit 1350 may be built into the space-floating image display device 1000. In cases where it is desired to more freely position the aerial operation detection sensor 1351 relative to the installation position of the space-floating image display device 1000, there is an advantage to a configuration in which only the aerial operation detection sensor 1351 is a separate unit.

[0076] The imaging unit 1180 is a camera with an image sensor, and captures images of the space near the floating-in-space image 3 and / or the face, arms, fingers, etc. of the user 230. A plurality of imaging units 1180 may be provided. For example, the imaging unit 1180 may be provided as a stereo camera. By using a plurality of imaging units 1180, or by using an imaging unit with a depth sensor, the mid-air operation detection unit 1350 can be assisted in detecting the touch operation of the floating-in-space image 3 by the user 230. The imaging unit 1180 may be provided separately from the floating-in-space image display device 1000. When the imaging unit 1180 is provided separately from the floating-in-space image display device 1000, it is sufficient to configure it so that an imaging signal can be transmitted to the floating-in-space image display device 1000 via a wired or wireless communication connection path, etc.

[0077] For example, if the aerial operation detection sensor 1351 is configured as an object intrusion sensor that targets a plane (intrusion detection plane) that includes the display surface (display range) of the spatial floating image 3 and detects whether or not an object has intruded into this intrusion detection plane, the aerial operation detection sensor 1351 may not be able to detect information such as how far an object that has not intruded into the intrusion detection plane (for example, a user's finger) is from the intrusion detection plane, or how close the object is to the intrusion detection plane.

[0078] In such a case, the distance between the object and the intrusion detection plane (Floating in Space Image 3) can be calculated by using information such as object depth calculation information based on the captured images of the multiple imaging units 1180 and object depth information from the depth sensor. Then, various information such as the depth calculation information, depth information, and distance between the object and the intrusion detection plane is used for various display controls for the Floating in Space Image 3.

[0079] Furthermore, without using the aerial operation detection sensor 1351, the aerial operation detection unit 1350 may detect a touch operation of the floating-in-space image 3 by the user 230 based on the captured image of the imaging unit 1180. In this case, the imaging unit 1180 may be referred to as an aerial operation detection sensor.

[0080] Furthermore, the imaging unit 1180 may capture an image of the face of the user operating the space-floating image 3, and the control unit 1110 or the like may perform a process to identify the user. Furthermore, in order to determine whether or not there is another person standing around or behind the user operating the space-floating image 3 and peeking at the user's operation of the space-floating image 3, the imaging unit 1180 may capture an image of the user operating the space-floating image 3 and a range including the user's surrounding area.

[0081] The operation input unit 1107 is, for example, an operation button, a signal receiving unit such as a remote controller, or an infrared light receiving unit, and inputs signals for operations different from the user's aerial operations (touch operations). Apart from the above-mentioned user touching the space-floating image 3, the operation input unit 1107 may also be used by, for example, an administrator to operate the space-floating image display device 1000.

[0082] The video signal input unit 1131 connects to an external video output device and inputs video data (video signals). The video signal input unit 1131 can be implemented using various digital video input interfaces. For example, the video signal input unit 1131 may be configured with a video input interface conforming to the HDMI (registered trademark) (High-Definition Multimedia Interface) standard, a video input interface conforming to the DVI (Digital Visual Interface) standard, or a video input interface conforming to the DisplayPort standard. Alternatively, an analog video input interface such as analog RGB or composite video may be provided.

[0083] The audio signal input unit 1133 is connected to an external audio output device and inputs audio data (audio signals). The audio signal input unit 1133 may be configured as an HDMI-standard audio input interface, an optical digital terminal interface, a coaxial digital terminal interface, or the like. In the case of an HDMI-standard interface, the video signal input unit 1131 and the audio signal input unit 1133 may be configured as an interface in which a terminal and a cable are integrated.

[0084] The audio output unit 1140 can output audio based on audio data input to the audio signal input unit 1133. The audio output unit 1140 may be configured with a speaker 1140. The audio output unit 1140 may be provided with a section that performs voice synthesis processing, etc. The audio output unit 1140 may also output built-in operation sounds and error warning sounds. Alternatively, the audio output unit 1140 may be configured to output audio as a digital signal to an external device, like the Audio Return Channel function defined in the HDMI standard.

[0085] The audio input unit 1139 may be configured with a microphone 1139. The microphone 1139 is a microphone that collects sounds around the space-floating image display device 1000, converts them into signals, and generates audio signals. The microphone may record a person's voice, such as a user's voice, and the control unit 1110 or the like performs voice recognition processing on the generated audio signal to obtain text information from the audio signal. The audio input unit 1139 may be provided with a part that performs voice recognition processing or the like. Note that the audio output unit 1140, the audio input unit 1139, etc. may be connected as external devices to the space-floating image display device 1000.

[0086] The non-volatile memory 1108 stores various data used by the space floating image display device 1000. The data stored in the non-volatile memory 1108 includes, for example, data for various operations to be displayed on the space floating image 3, display icons, data of objects for the user to operate, layout information, etc. The memory 1109 stores image data to be displayed as the space floating image 3, data for controlling the device, etc.

[0087] The control unit 1110 includes a processor and controls the operation of each connected unit. The control unit 1110 may also work in cooperation with a program stored in the memory 1109 to perform calculations based on information acquired from each unit in the space floating image display device 1000.

[0088] The communication unit 1132 communicates with external devices, external servers, etc. via a wired or wireless communication interface. If the communication unit 1132 has a wired communication interface, the wired communication interface may be configured, for example, by an Ethernet-standard LAN interface. If the communication unit 1132 has a wireless communication interface, the interface may be configured, for example, by a Wi-Fi communication interface, a Bluetooth communication interface, or a mobile communication interface such as 4G or 5G. Various types of data, such as video data, image data, and audio data, are transmitted and received through communication via the communication unit 1132.

[0089] The removable media interface 1134 is an interface for connecting a removable recording medium (removable media). The removable recording medium (removable media) may be composed of a semiconductor device memory such as a solid state drive (SSD), a magnetic recording medium recording device such as a hard disk drive (HDD), or an optical recording medium such as an optical disk. The removable media interface 1134 is capable of reading various information, such as video data, image data, and audio data, recorded on the removable recording medium. The video data, image data, etc. recorded on the removable recording medium are output as the floating image 3 via the video display unit 1102 and the retroreflection unit 1101.

[0090] The storage unit 1170 is a storage device that records various types of information, such as video data, image data, and audio data. The storage unit 1170 may be configured with a magnetic recording medium recording device, such as a hard disk drive (HDD), or a semiconductor element memory, such as a solid state drive (SSD). For example, various types of information, such as video data, image data, and audio data, may be recorded in advance in the storage unit 1170 at the time of product shipment. The storage unit 1170 may also record various types of information, such as video data, image data, and audio data, acquired from an external device, an external server, or the like via the communication unit 1132.

[0091] The video data, image data, etc. recorded in the storage unit 1170 are output as the space floating image 3 via the video display unit 1102 and the retroreflection unit 1101 based on processing by the video control unit 1160. Video data, image data, etc. of display icons, objects for user operation, etc. displayed as the space floating image 3 are also recorded in the storage unit 1170. Layout information of the display icons, objects, etc. displayed as the space floating image 3, and various metadata information related to the objects, etc. are also recorded in the storage unit 1170.

[0092] The audio data recorded in the storage unit 1170 is output as audio from the audio output unit 1140, for example.

[0093] The video control unit 1160 performs various controls related to the video signal input to the video display unit 1102. Based on the video signal (video data), the video control unit 1160 creates a video signal (display data) for displaying a video on the video display unit 1102 (e.g., the liquid crystal display panel 11 of the display device 1 described above), and supplies the video signal to the video display unit 1102. The video control unit 1160 may also be referred to as a video processing circuit, and may be configured with hardware such as an ASIC, FPGA, or video processor. The video control unit 1160 may also be referred to as a video processing unit or image processing unit. The video control unit 1160 performs video switching control, such as determining which video signal to input to the video display unit 1102, between the video signal to be stored in the memory 1109 and the video signal (video data) input to the video signal input unit 1131.

[0094] Note that control unit 1110 may perform the same processing as video control unit 1160, in which case control unit 1110 may be referred to as a video processing unit, etc. At least one of control unit 1110, video control unit 1160, aerial operation detection unit 1360, etc. may perform unique control processing, in which case control unit 1110, video control unit 1160, aerial operation detection unit 1360, etc. may be referred to as a video processing unit.

[0095] In addition, the image control unit 1160 may generate a superimposed image signal by superimposing the image signal to be stored in the memory 1109 and the image signal input from the image signal input unit 1131, and input the superimposed image signal to the image display unit 1102, thereby controlling the formation of a composite image as a floating image in space 3.

[0096] Furthermore, the video control unit 1160 may control image processing of the video signal input from the video signal input unit 1131, the video signal to be stored in the memory 1109, etc. Examples of image processing include scaling processing that enlarges, reduces, or deforms the image, brightness adjustment processing that changes the brightness, contrast adjustment processing that changes the contrast curve of the image, and Retinex processing that decomposes the image into light components and changes the weighting of each component.

[0097] Furthermore, the video control unit 1160 may perform special effect video processing or the like to assist the user's aerial operation (touch operation) on the video signal input to the video display unit 1102. The special effect video processing is performed, for example, based on the detection result of the user's touch operation by the aerial operation detection unit 1350, or on an image of the user captured by the imaging unit 1180. Furthermore, the video control unit 1160 or the like may perform audio control processing when audio is output from the audio output unit 1140 simultaneously with the floating-in-space image 3. An audio control unit for this audio control processing may be provided separately from the video control unit 1160.

[0098] The attitude sensor 1113 is a sensor configured with a gravity sensor, an acceleration sensor, or a combination of these, and can detect the attitude in which the space-floating image display device 1000 is installed. Based on the attitude detection result of the attitude sensor 1113, the control unit 1110 may control the operation of each connected unit. For example, if an undesirable attitude is detected as the user's usage state, the control unit 1110 may perform control such that the image displayed on the image display unit 1102 is stopped and an error message is displayed to the user. Alternatively, if the attitude sensor 1113 detects a change in the installation attitude of the space-floating image display device 1000, the control unit 1110 may perform control such that the display orientation of the image displayed on the image display unit 1102 is rotated.

[0099] As explained above, various functions are installed in the space floating image display device 1000. However, the space floating image display device 1000 does not need to have all of these functions, and any configuration is acceptable as long as it has the function of forming the space floating image 3.

[0100] <Configuration Example of Space-Floating Image Display Device> Next, a configuration example of the space-floating image display device will be described. The layout of the components of the space-floating image display device according to this embodiment can be various depending on the usage form. Below, the layouts of each of Figs. 4A to 4P will be described. Note that in each of the examples of Figs. 4A to 4P, the thick lines surrounding the components (display device 1, etc.) of the space-floating image display device 1000 indicate an example of the housing structure (housing 1190 in Fig. 3) of the space-floating image display device 1000.

[0101] FIG. 4A is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4A is equipped with an optical system corresponding to the optical system shown in FIG. 2A. The space-floating image display device 1000 shown in FIG. 4A is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4A, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. When the mid-air operation detection sensor 1351 is installed as shown in the figure, it can detect the operation of the space-floating image 3 by the user's finger. Note that the x direction is the left-right direction as seen from the user, the y direction is the front-back direction (depth direction) as seen from the user, and the z direction is the up-down direction (vertical direction). Hereinafter, the definitions of the x direction, y direction, and z direction are the same in each drawing of FIG. 4, so repeated explanations will be omitted.

[0102] FIG. 4B is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4B is equipped with an optical system corresponding to the optical system of FIG. 2A. The space-floating image display device 1000 shown in FIG. 4B is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (toward the user 230). That is, in FIG. 4B, the space-floating image display device is installed with the transparent member 100 facing the front of the device (toward the user 230). The space-floating image 3 is formed on the user 230 side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. When the mid-air operation detection sensor 1351 is installed as shown in the figure, it is possible to detect the operation of the space-floating image 3 by the user 230's finger. 4B , the aerial operation detection sensor 1351 can use the reflection of sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, the reflectivity of the nail is higher than that of the finger pad, so that this configuration can improve the accuracy of touch detection.

[0103] FIG. 4C is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4C is equipped with an optical system corresponding to the optical system of FIG. 2B. The space-floating image display device 1000 shown in FIG. 4C is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4C, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. When the mid-air operation detection sensor 1351 is installed as shown in the figure, it is possible to detect the operation of the space-floating image 3 by the user's 230 finger.

[0104] FIG. 4D is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4D is equipped with an optical system corresponding to the optical system of FIG. 2B. The space-floating image display device 1000 shown in FIG. 4D is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (toward the user 230). That is, in FIG. 4D, the space-floating image display device 1000 is installed with the transparent member 100 facing the front of the device (toward the user 230). The space-floating image 3 is formed on the user 230 side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. If the mid-air operation detection sensor 1351 is installed as shown in the figure, it can detect the operation of the space-floating image 3 by the user 230's finger. 4D , the aerial operation detection sensor 1351 can use the reflection of sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, the reflectivity of the nail is higher than that of the finger pad, so that this configuration can improve the accuracy of touch detection.

[0105] FIG. 4E is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4E is equipped with an optical system corresponding to the optical system of FIG. 2C. The space-floating image display device 1000 shown in FIG. 4E is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward. That is, in FIG. 4E, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels directly upward. When the mid-air operation detection sensor 1351 is provided as shown in the figure, it is possible to detect the operation of the space-floating image 3 by the user's 230 finger.

[0106] FIG. 4F is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4F is equipped with an optical system corresponding to the optical system of FIG. 2C. The space-floating image display device 1000 shown in FIG. 4F is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (toward the user 230). That is, in FIG. 4F, the space-floating image display device 1000 is installed with the transparent member 100 facing the front of the device (toward the user 230). The space-floating image 3 is formed on the user 230 side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels toward the user. If the mid-air operation detection sensor 1351 is installed as shown in the figure, it can detect the operation of the space-floating image 3 by the user 230's finger.

[0107] FIG. 4G is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4G is equipped with an optical system corresponding to the optical system shown in FIG. 2C. In the optical systems of the space-floating image display devices shown in FIGS. 4A to 4F, the central optical path of the image light emitted from the display device 1 was on the yz plane. That is, in the optical systems of the space-floating image display devices shown in FIGS. 4A to 4F, the image light traveled in the front-to-back and up-to-down directions as seen from the user. In contrast, in the optical system of the space-floating image display device shown in FIG. 4G, the central optical path of the image light emitted from the display device 1 is on the xy plane. That is, in the optical system of the space-floating image display device shown in FIG. 4G, the image light travels in the left-to-right and front-to-back directions as seen from the user. In the space-floating image display device 1000 shown in FIG. 4G, the surface on which the space-floating image 3 is formed is installed so that it faces the front of the device (toward the user 230). 4G, the space-floating image display device 1000 has the transparent member 100 installed on the front side of the device (toward the user 230). The space-floating image 3 is formed on the user side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels towards the user. If the mid-air operation detection sensor 1351 is installed as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230.

[0108] 4H is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4H differs from the space-floating image display device of FIG. 4G in that it has a window with a transparent plate 100B made of glass or plastic on the back of the device (opposite the position where the user 230 views the space-floating image 3, i.e., opposite the traveling direction of the image light of the space-floating image 3 toward the user 230). The rest of the configuration is the same as the space-floating image display device of FIG. 4G , so repeated explanations will be omitted. The space-floating image display device 1000 of FIG. 4H has a window with a transparent plate 100B on the opposite side of the traveling direction of the image light of the space-floating image 3 from the space-floating image 3. Therefore, when the user 230 views the space-floating image 3, they can recognize the scenery behind the space-floating image display device 1000 as the background of the space-floating image 3. Therefore, the user 230 can perceive the space floating image 3 as floating in the air in front of the scenery behind the space floating image display device 1000. This can further emphasize the floating feeling of the space floating image 3.

[0109] Depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, a portion of the image light output from the display device 1 may be reflected by the polarization separation member 101B and head toward the transparent plate 100B. Depending on the coating performance of the surface of the transparent plate 100B, this light may be reflected again by the surface of the transparent plate 100B and may be visually recognized by the user 230 as stray light. Therefore, in order to prevent this stray light, the transparent plate 100B may not be provided in the window on the back of the space-floating image display device 1000.

[0110] Fig. 4I is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4I differs from the space-floating image display device of Fig. 4H in that a light-blocking door 1410 is provided in the window of the transparent plate 100B located on the back of the device (opposite the position where the user 230 views the space-floating image 3). The rest of the configuration is the same as that of the space-floating image display device of Fig. 4H, so repeated explanations will be omitted.

[0111] The opening and closing door 1410 of the space-floating image display device 1000 in FIG. 4I has, for example, a light blocking plate and is equipped with a mechanism for moving (sliding), rotating, or attaching / detaching the light blocking plate, thereby switching between an open state and a light blocking state for the window (rear window) of the transparent plate 100B located at the back of the space-floating image display device 1000. The movement (sliding) and rotation of the light blocking plate by the opening and closing door 1410 may be electrically driven by a motor (not shown). The motor may be controlled by the control unit 1110 in FIG. 3. Note that the example in FIG. 4I discloses an example in which the opening and closing door 1410 has two light blocking plates. In contrast, the opening and closing door 1410 may have only one light blocking plate.

[0112] For example, if the view seen through the window of the transparent plate 100B of the space-floating image display device 1000 is outdoors, the brightness of sunlight varies depending on the weather. When the outdoor sunlight is strong, the background of the space-floating image 3 may become too bright, reducing the user 230's visibility of the space-floating image 3. In such a case, by moving (sliding), rotating, or attaching the light-shielding plate of the opening / closing door 1410 to block the light from the rear window, the background of the space-floating image 3 becomes dark, thereby relatively increasing the visibility of the space-floating image 3. Such a blocking operation by the light-shielding plate of the opening / closing door 1410 may be performed directly by the force of the user 230's hand. In response to an operation input via the operation input unit 1107 of FIG. 3 , the control unit 1110 may control a motor (not shown) to perform the blocking operation by the light-shielding plate of the opening / closing door 1410.

[0113] An illuminance sensor may be provided on the rear side (opposite the user 230) of the space-floating image display device 1000, such as near the rear window, to measure the brightness of the space beyond the rear window. In this case, the control unit 1110 of Fig. 3 may control a motor (not shown) to perform the opening and closing operation of the light blocking plate of the opening and closing door 1410 according to the detection result of the illuminance sensor. By controlling the opening and closing operation of the light blocking plate of the opening and closing door 1410 in this way, it is possible to more suitably maintain the visibility of the space-floating image 3, even if the user 230 does not manually open or close the light blocking plate of the opening and closing door 1410.

[0114] Furthermore, the light blocking plate provided by the opening and closing door 1410 may be manually detachable. Depending on the intended use and installation environment of the space floating image display device 1000, the user can select whether to leave the rear window open or in a light blocking state. If it is planned to use the rear window in a light blocking state for a long period of time, the detachable light blocking plate can be fixed in the light blocking state. Furthermore, if it is planned to use the rear window in an open state for a long period of time, the detachable light blocking plate can be left detached. The light blocking plate may be attached and detached using screws, a hook structure, or a fitting structure.

[0115] 4I, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, a portion of the image light output from the display device 1 may be reflected by the polarization separation member 101B and directed toward the transparent plate 100B. Depending on the coating performance of the surface of the transparent plate 100B, this light may be reflected again by the surface of the transparent plate 100B and be visible to the user 230 as stray light. Therefore, in order to prevent this stray light, the window on the back of the space-floating image display device 1000 may not be provided with the transparent plate 100B. The above-described opening / closing door 1410 may be provided in a window that does not have the transparent plate 100B. To prevent this stray light, it is desirable that the inner surface of the housing of the light-shielding plate of the above-described opening / closing door 1410 have a coating or material with low light reflectance.

[0116] FIG. 4J is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4J differs from the space-floating image display device of FIG. 4H in that an electronically controlled transmittance variable device 1620 is disposed on the rear side window instead of the transparent plate 100B made of glass or plastic. The other components are the same as those of the space-floating image display device of FIG. 4H , so repeated explanations will be omitted. An example of the electronically controlled transmittance variable device 1620 is a liquid crystal shutter. Although the electronically controlled transmittance variable device 1620 is not shown in FIG. 3 , if it is provided, it may be configured as one component of the space-floating image display device 1000 of FIG. 3 , and be connected to other processing units such as the control unit 1110.

[0117] The liquid crystal shutter can control the light transmittance by controlling the voltage of the liquid crystal element sandwiched between two polarizing plates. Therefore, if the liquid crystal shutter is controlled to increase the transmittance, the scenery through the rear window can be seen through the background of the floating image 3. On the other hand, if the liquid crystal shutter is controlled to decrease the transmittance, the scenery through the rear window can be hidden as the background of the floating image 3.

[0118] Furthermore, since the liquid crystal shutter can control halftones, it can also be set to a state of transmittance of 50% or the like. For example, the control unit 1110 can control the transmittance of the electronically controlled transmittance variable device 1620 in response to an operation input via the operation input unit 1107 in Fig. 3. With this configuration, in cases where a viewer wants to see the scenery through the rear window as the background of the Space Floating Image 3, but the scenery through the rear window as the background is too bright and reduces the visibility of the Space Floating Image 3, it is possible to adjust the transmittance of the electronically controlled transmittance variable device 1620 and thereby adjust the visibility of the Space Floating Image 3.

[0119] In addition, an illuminance sensor may be provided on the rear side (opposite the user 230) of the space-floating image display device 1000, such as near the rear window, to measure the brightness of the space beyond the rear window. In this case, the control unit 1110 of Fig. 3 controls the transmittance of the electronically controlled transmittance variable device 1620 according to the detection result of the illuminance sensor. In this way, even if the user 230 does not perform an operation input via the operation input unit 1107 of Fig. 3, the transmittance of the electronically controlled transmittance variable device 1620 can be adjusted according to the brightness of the space beyond the rear window, making it possible to more suitably maintain the visibility of the space-floating image 3.

[0120] In the above example, a liquid crystal shutter has been described as an example of the electronically controlled transmittance varying device 1620. However, electronic paper may be used as another example of the electronically controlled transmittance varying device 1620. The same effect as described above can be obtained when electronic paper is used. Furthermore, electronic paper consumes very little power to maintain a halftone state. Therefore, a space floating image display device with lower power consumption can be realized compared to when a liquid crystal shutter is used.

[0121] Fig. 4K is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4K differs from the space-floating image display device of Fig. 4G in that it has a transmissive self-luminous image display device 1650 instead of the transparent member 100. The other configurations are the same as those of the space-floating image display device of Fig. 4G, so repeated explanations will be omitted.

[0122] In the space-floating image display device 1000 of FIG. 4K, an image luminous flux passes through the display surface of the transmissive self-luminous image display device 1650, and then a space-floating image 3 is formed outside the space-floating image display device 1000. That is, when an image is displayed on the transmissive self-luminous image display device 1650, which is a two-dimensional flat display, the space-floating image 3 can be displayed as a pop-up image further in front of the image displayed on the transmissive self-luminous image display device 1650. In this case, the user 230 can simultaneously view two images at different depth positions. The transmissive self-luminous image display device 1650 may be configured using existing technology, such as a transmissive organic EL panel, as disclosed in, for example, Japanese Patent Application Laid-Open No. 2014-216761. Incidentally, when the transmissive self-luminous image display device 1650 is provided, it may be configured to be connected to other processing units, such as the control unit 1110, as a component of the space-floating image display device 1000 of FIG. 3.

[0123] Here, if the transmissive self-luminous image display device 1650 displays both the background and an object such as a character, and then displays only the object such as the character moving into the floating image 3 in front, it is possible to provide the user 230 with a more effective surprise-like image experience.

[0124] Furthermore, if the interior of the space-floating image display device 1000 (housing 1190) is kept light-shielded, the background of the transmissive self-luminous image display device 1650 will be sufficiently dark. Therefore, when no image is displayed on the display device 1 or the light source of the display device 1 is turned off and an image is displayed only on the transmissive self-luminous image display device 1650, the transmissive self-luminous image display device 1650 appears to the user 230 as a normal two-dimensional flat display, rather than a transmissive display. Note that, since the space-floating image 3 in the embodiment of the present invention is displayed as a real optical image in a space without a screen, if the light source of the display device 1 is turned off, the intended display position of the space-floating image 3 becomes empty space. Therefore, when the transmissive self-luminous image display device 1650 is used to display an image as if it were a normal two-dimensional flat display, characters, objects, etc., can be suddenly displayed in the air as the space-floating image 3, providing the user 230 with a more effective and surprising video experience.

[0125] Note that the darker the interior of the space-floating image display device 1000, the more the transmissive self-luminous image display device 1650 appears like a two-dimensional flat display. Therefore, an absorptive polarizer (not shown) that transmits the polarized light of the image light reflected by the polarization separation member 101B and absorbs polarized light that is 90° out of phase with the polarized light may be provided on the interior surface of the transmissive self-luminous image display device 1650 (the surface on which the image light reflected by the polarization separation member 101B enters the transmissive self-luminous image display device 1650, i.e., the surface of the transmissive self-luminous image display device 1650 opposite the space-floating image 3). While this does not have a significant effect on the image light that forms the space-floating image 3, it can significantly reduce the light that enters the space-floating image display device 1000 from the outside through the transmissive self-luminous image display device 1650, making the interior of the space-floating image display device 1000 darker, which is preferable.

[0126] 4L is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of FIG. 4L is a modified example of the space-floating image display device of FIG. 4K. The orientation of the components in the space-floating image display device 1000 is different from that of the space-floating image display device of FIG. 4K, and is closer to the arrangement of the space-floating image display device of FIG. 4F. The functions and operations of each component are the same as those of the space-floating image display device of FIG. 4K, so repeated explanations will be omitted.

[0127] In the space-floating image display device of FIG. 4L, after the luminous flux of image light passes through the transmissive self-luminous image display device 1650, a space-floating image 3 is formed on the user 230 side of the transmissive self-luminous image display device 1650.

[0128] In both the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L, the space-floating image 3 is displayed superimposed on the image of the transmissive self-luminous image display device 1650 as seen by the user 230. Here, the position of the space-floating image 3 and the position of the image of the transmissive self-luminous image display device 1650 are configured to have a difference in the depth direction. Therefore, when the user 230 moves his / her head (position of viewpoint), the depth of the two images can be recognized by parallax. Therefore, by displaying two images at different depth positions, it is possible to provide the user with a more suitable three-dimensional image experience with the naked eye without the need for stereoscopic glasses or the like.

[0129] Fig. 4M is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4M has a second display device 1680 provided on the far side (for example, the back panel of the housing 1190) as seen from the user 230 relative to the polarization separation member 101B of the space-floating image display device of Fig. 4G. The other configurations are the same as those of the space-floating image display device of Fig. 4G, so repeated explanations will be omitted.

[0130] In the configuration example shown in Fig. 4M, the second display device 1680 is provided behind the display position of the space-floating image 3, and the image display surface is directed toward the space-floating image 3. With this configuration, from the user 230's perspective, the image of the second display device 1680 and the image of the space-floating image 3, which are displayed at two different depth positions, can be viewed superimposed on each other. In other words, the second display device 1680 can be said to be arranged so as to display an image toward the user 230 who views the space-floating image 3. When the second display device 1680 is provided, it may be configured to be connected to other processing units such as the control unit 1110 as one component of the space-floating image display device 1000 of Fig. 3.

[0131] Note that the image light of the second display device 1680 of the space-floating image display device 1000 of FIG. 4M is viewed by the user 230 after passing through the polarization separation member 101B. Therefore, in order for the image light of the second display device 1680 to more suitably pass through the polarization separation member 101B, it is desirable that the image light output from the second display device 1680 be polarized with a polarization in a vibration direction that the polarization separation member 101B more suitably transmits. In other words, it is desirable that the image light be polarized with a polarization in the same vibration direction as the polarization of the image light output from the display device 1. For example, if the image light output from the display device 1 is S-polarized, it is desirable that the image light output from the second display device 1680 is also S-polarized. Furthermore, if the image light output from the display device 1 is P-polarized, it is desirable that the image light output from the second display device 1680 is also P-polarized.

[0132] The example of the space-floating image display device of FIG. 4M also has the same effect as the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L in that a second image is displayed behind the space-floating image 3. However, unlike the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L, in the example of the space-floating image display device of FIG. 4M, the luminous flux of image light for forming the space-floating image 3 does not pass through the second display device 1680. Therefore, the second display device 1680 does not need to be a transmissive self-luminous image display device, but may be a liquid crystal display, which is a two-dimensional flat display. The second display device 1680 may also be an organic EL display. Therefore, in the example of the space-floating image display device of FIG. 4M, it is possible to realize the space-floating image display device 1000 at a lower cost than the example of the space-floating image display device of FIG. 4K and the example of the space-floating image display device of FIG. 4L.

[0133] Here, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, there is a possibility that part of the image light output from the display device 1 will be reflected by the polarization separation member 101B and head toward the second display device 1680. This light (part of the image light) may be reflected again by the surface of the second display device 1680 and may be visually recognized by the user as stray light.

[0134] Therefore, to prevent this stray light, an absorptive polarizer may be provided on the surface of the second display device 1680. In this case, the absorptive polarizer may be an absorptive polarizer that transmits the polarized waves of the image light output from the second display device 1680 and absorbs polarized waves that are 90° out of phase with the polarized waves of the image light output from the second display device 1680. If the second display device 1680 is a liquid crystal display, an absorptive polarizer is also present on the image output side inside the liquid crystal display. However, if a cover glass (cover glass on the image display surface side) is further provided on the output surface of the absorptive polarizer on the image output side inside the liquid crystal display, it is not possible to prevent stray light caused by reflection of the cover glass by light from outside the liquid crystal display. Therefore, it is necessary to separately provide the above-mentioned absorptive polarizer on the surface of the cover glass.

[0135] When an image is displayed on the second display device 1680, which is a two-dimensional flat display, the floating-in-space image 3 can be displayed as an image further in front of the user in relation to the image on the second display device 1680. In this case, the user 230 can simultaneously view two images at different depth positions. By displaying a character on the floating-in-space image 3 and a background on the second display device 1680, it is possible to provide the effect that the user 230 is viewing the space in which the character exists in three dimensions.

[0136] Furthermore, if the second display device 1680 displays both the background and an object such as a character, and then displays only the object such as the character moving into the floating image 3 in front, it is possible to provide the user 230 with a more effective surprise visual experience.

[0137] Next, Fig. 4N is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 of Fig. 4N is a space-floating image display device that employs the optical system of Fig. 2D. In the space-floating image display device 1000 of Fig. 4N, similar to the examples of the space-floating image display device that employ the optical systems of Figs. 2A to 2C, image light that has passed through a transparent member 100 is formed in the air as a space-floating image 3. In addition, using sensing light from an aerial operation detection sensor 1351 that is located on the back side of the transparent member 100 as seen from the user, it is possible to detect operation of the space-floating image 3 by the user's finger 9004.

[0138] In both the example of the space-floating image display device employing the optical systems of FIGS. 2A to 2C and the example of the space-floating image display device employing the optical system of FIG. 2D, the space-floating image 3 is formed in front of the transparent member 100, and the operation of the space-floating image 3 by the user's finger can be detected using the sensing light of the mid-air operation detection sensor 1351 located on the back side of the transparent member 100 as seen by the user.

[0139] The space-floating image display device employing the optical system of Fig. 2D has a different optical system from the space-floating image display device in which the optical system of Fig. 2A to Fig. 2C is arranged on the back side of the transparent member 100 as seen from the user. However, the usability of the space-floating image display device employing the optical system of Fig. 2D as seen from the user is almost the same as that of the space-floating image display device employing the optical system of Fig. 2A to Fig. 2C.

[0140] Next, Fig. 4O is a diagram showing an example of the configuration of a space-floating image display device. Fig. 4O is a diagram showing the configuration of the internal optical system of the space-floating image display device 1000 of Fig. 4N. The space-floating image display device 1000 shown in Fig. 4O is equipped with an optical system corresponding to the optical system of Fig. 2D. The space-floating image display device 1000 shown in Fig. 4O is installed horizontally so that the surface on which the space-floating image 3 is formed faces upward.

[0141] 4O, the space-floating image display device 1000 has a transparent member 100 installed on the top surface of the device. The space-floating image 3 is formed above the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. When the mid-air operation detection sensor 1351 is provided as shown in the figure, it can detect the operation of the space-floating image 3 by the finger of the user 230.

[0142] Here, the configuration of Fig. 4O will be compared with the configuration of Fig. 4A to confirm the differences. In Fig. 4A, the display device 1 and the space-floating image 3 are in a plane-symmetrical relationship with respect to the plane of the polarization separation member 101. In contrast, in Fig. 4O, the display device 1 and the space-floating image 3 are in a plane-symmetrical relationship with respect to the plane of the retroreflector 5. Furthermore, the configuration of Fig. 4A includes the retroreflector 2 and the λ / 4 plate 21, but these are not present in Fig. 4O. Furthermore, while the presence of the absorbing polarizer 12 is more preferable in Fig. 4A, the absorbing polarizer 12 is not particularly necessary in Fig. 4O.

[0143] To replace the optical system of FIG. 2A in the configuration of FIG. 4A with the optical system of FIG. 2D and convert it into the configuration of FIG. 4O, the following can be done. That is, the polarization separation member 101 in the configuration of FIG. 4A is replaced with the retroreflector 5, and the retroreflector 2 and the λ / 4 plate 21 are removed from the configuration of FIG. 4A. The absorptive polarizer 12 is optional. By making a substitution based on this idea, the optical system of FIGS. 2A to 2C installed in the configuration of the space-floating image display device of FIGS. 4A to 4G can be replaced with the optical system of FIG. 2D, and the space-floating image display device can be replaced with the optical system of FIG. 2D. In this case, the polarization separation member 101 in FIGS. 4A and 4B is replaced with the retroreflector 5, and the polarization separation member 101B in FIGS. 4C to 4G is replaced with the retroreflector 5.

[0144] For example, FIG. 4P is a diagram showing an example of the configuration of a space-floating image display device. The space-floating image display device 1000 shown in FIG. 4P is equipped with an optical system corresponding to the optical system shown in FIG. 2D. FIG. 4P shows the configuration of the space-floating image display device of FIG. 4B, with the optical system shown in FIG. 2A replaced with the optical system shown in FIG. 2D. The space-floating image display device 1000 shown in FIG. 4P is installed vertically so that the surface on which the space-floating image 3 is formed faces the front of the space-floating image display device 1000 (toward the user 230). That is, in FIG. 4P, the space-floating image display device is installed with the transparent member 100 facing the front of the device (toward the user 230). The space-floating image 3 is formed on the user 230 side of the surface of the transparent member 100 of the space-floating image display device 1000. The light of the space-floating image 3 travels diagonally upward. When the mid-air operation detection sensor 1351 is installed as shown in the figure, it is possible to detect the operation of the space-floating image 3 by the user 230's finger. 4P, the aerial operation detection sensor 1351 can use the reflection of sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, the reflectivity of the nail is higher than that of the finger pad, so that this configuration can improve the accuracy of touch detection.

[0145] According to the configuration of the space floating image display device of FIGS. 4N to 4P, it is possible to realize a user-friendly space floating image display device using the optical system of FIG. 2D.

[0146] <Display Device> Next, the display device 1 of this embodiment will be described with reference to the drawings. The display device 1 of this embodiment includes a liquid crystal display panel 11 as an image display element 11, and a light source device 13 that constitutes a light source for the liquid crystal display panel 11. In Fig. 5, the light source device 13 is shown together with the liquid crystal display panel 11 as an exploded perspective view.

[0147] As shown by arrow 30 in Fig. 5, the image display element 11, which is a liquid crystal display panel, receives an illumination light beam from a light source device 13, which is a backlight device, that has narrow-angle diffusion characteristics, i.e., has strong directionality (in other words, linearity) and characteristics similar to laser light with a polarization plane aligned in one direction. The image display element 11, which is a liquid crystal display panel 11, modulates the received illumination light beam in accordance with an input video signal. The modulated image light is reflected by the retroreflector 2 and passes through the transparent member 100 to form a real image, a floating image (see Fig. 1).

[0148] 5, the display device 1 is configured with a light source device 13 and a liquid crystal display panel 11, a light redirection panel 54 that controls the directional characteristics of the light beam emitted from the light source device 13, and, if necessary, a narrow-angle diffuser (not shown). That is, polarizing plates are provided on both sides of the liquid crystal display panel 11, and as indicated by arrow 30 in FIG. 5, image light of a specific polarization is emitted with its intensity modulated by a video signal. This allows the desired image to be projected as highly directional (linear) light of a specific polarization via the light redirection panel 54 toward the retroreflector 2, which then reflects the light toward the eyes of a monitor outside the store (space) shown in FIG. 1, forming the floating image 3. A protective cover 50 (see FIGS. 6 and 7) may be provided on the surface of the light redirection panel 54.

[0149] <Display Device Example 1> FIG. 6 shows an example of a specific configuration of the display device 1. In FIG. 6, a liquid crystal display panel 11 and a light direction conversion panel 54 are arranged on the light source device 13 shown in FIG. 5. This light source device 13 is configured, for example, by housing LED elements 201 and a light guide 203 inside a plastic case or the like. As shown in FIG. 5 and other figures, the end surface of the light guide 203 has a lens-like shape whose cross-sectional area gradually increases toward the light receiving section in order to convert the divergent light from each LED element 201 into a substantially parallel beam. The lens-like shape gradually reduces the divergence angle through multiple total reflections as the light propagates through the light guide 203. The liquid crystal display panel 11 constituting the display device 1 is attached to the top surface of the display device 1. Furthermore, an LED substrate 202 mounting the LED elements 201, which are semiconductor light sources, and their associated control circuits is attached to one side surface of the light source device 13 (the left end surface in this example). A heat sink, a component for cooling the heat generated by the LED elements 201 and the control circuit, may be attached to the outer surface of the LED substrate 202.

[0150] Furthermore, a frame (not shown) for the liquid crystal display panel 11 is attached to the top surface of the case of the light source device 13. The frame (not shown) for the liquid crystal display panel 11 is also attached to the frame, and an FPC (Flexible Printed Circuit) (not shown) electrically connected to the liquid crystal display panel 11 is also attached. That is, the liquid crystal display panel 11, which is the image display element 11, generates a display image by modulating the intensity of transmitted light in conjunction with the LED elements 201, which are solid-state light sources, based on a control signal from a control circuit (image control unit 1160 in FIG. 3 ) constituting the electronic device. The generated image light has a narrow diffusion angle and contains only specific polarization components, resulting in a novel image display device similar to a surface-emitting laser image source driven by a video signal. Currently, it is technically and safety-wise impossible to obtain a laser beam of the same size as the image obtained by the display device 1 described above using a laser device. Therefore, in this embodiment, light similar to the surface-emitting laser image light described above is obtained from a beam of light from a general light source, for example, an LED element.

[0151] Next, the configuration of the optical system housed in the case of light source device 13 will be described in detail with reference to Fig. 6 and Fig. 7. Because Fig. 6 and Fig. 7 are cross-sectional views, only one of the multiple LED elements 201 constituting the light source is shown, and this is converted into approximately parallel light (collimated light) by the shape of the light-receiving end surface 203a of the light guide 203. For this reason, the light-receiving portion of the light guide end surface and the LED element 201 are attached while maintaining a predetermined positional relationship.

[0152] Each light guide 203 is formed of a translucent resin such as acrylic. The LED light-receiving surface at the end of the light guide 203 has a cone-shaped outer periphery obtained by rotating a parabolic cross section, with a centrally-convex concave portion (i.e., a convex lens surface) at the top, and a centrally-convex convex lens surface (or a concave lens surface) at the flat surface (not shown). The outer shape of the light-receiving portion of the light guide to which the LED element 201 is attached is a parabolic shape forming a cone-shaped outer periphery, and is set within an angle range that allows total reflection of the light emitted from the LED element toward the periphery, or a reflective surface is formed.

[0153] On the other hand, the LED elements 201 are arranged at predetermined positions on the surface of the circuit board, that is, the LED substrate 202. The LED substrate 202 is arranged and fixed so that the LED elements 201 on the surface are positioned in the center of the recessed portion described above with respect to the light-receiving end surface 203a, that is, the LED collimator.

[0154] With this configuration, the shape of the light-receiving end surface 203a of the light guide 203 makes it possible to extract the light emitted from the LED element 201 as approximately parallel light, thereby improving the utilization efficiency of the generated light.

[0155] As described above, the light source device 13 is configured by attaching a light source unit having a plurality of LED elements 201 arranged as light sources to the light-receiving end surface 203a, which is a light-receiving section provided on the end surface of the light guide 203, and the divergent light beams from the LED elements 201 are converted into approximately parallel light by the lens shape of the light-receiving end surface 203a of the light guide 203, which is guided inside the light guide 203 as shown by the arrow, and is emitted by the light beam direction conversion means 204 toward the liquid crystal display panel 11, which is arranged approximately parallel to the light guide 203. By optimizing the distribution (in other words, density) of the light beam direction conversion means 204 depending on the shape of the inside or surface of the light guide 203, it is possible to control the uniformity of the light beam incident on the liquid crystal display panel 11.

[0156] The light beam direction conversion means 204 described above emits the light beam propagated inside the light guide 203 toward the liquid crystal display panel 11 disposed approximately parallel to the light guide 203, by using the shape of the surface of the light guide 203 or by providing a portion with a different refractive index inside the light guide 203. At this time, when the liquid crystal display panel 11 is faced directly at the center of the screen and the viewpoint is positioned at the same position as the diagonal dimension of the screen, and the brightness at the center of the screen and the brightness at the periphery of the screen are compared, if the relative brightness ratio is 20% or more, there is no practical problem, and if it exceeds 30%, it will be an even better characteristic.

[0157] 6 is a cross-sectional layout diagram illustrating the configuration and operation of a light source of this embodiment that performs polarization conversion in light source device 13 including light guide 203 and LED element 201 described above. In Fig. 6, light source device 13 is composed of light guide 203 formed of, for example, plastic or the like and having light beam direction conversion means 204 on its surface or inside, LED element 201 as a light source, reflective sheet 205, retardation plate 206, lenticular lens, etc. Attached to the top surface of light source device 13 is liquid crystal display panel 11 that has polarizing plates on the light source light entrance surface and the image light exit surface.

[0158] Furthermore, a film- or sheet-like reflective polarizing plate 49 is provided on the light source light incidence surface (the lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, selectively reflecting one polarized wave (e.g., P wave) 212 of the natural light beam 210 emitted from the LED element 201. The reflected light is reflected again by a reflective sheet 205 provided on one surface (the lower surface in the figure) of the light guide 203 and directed toward the liquid crystal display panel 11. Therefore, a retardation plate (λ / 4 plate) is provided between the reflective sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49. The reflected light (reflected light beam) is reflected by the reflective sheet 205 and passes through the retardation plate (λ / 4 plate) a total of two times, thereby converting it from P-polarized light to S-polarized light. This improves the utilization efficiency of the light source light as image light. The image light beam, the light intensity of which has been modulated by the image signal in the liquid crystal display panel 11, is emitted as shown by the arrow 213 in Fig. 6 and enters the retroreflector 2. After being reflected by the retroreflector 2, a real image, a floating image in space, can be obtained.

[0159] 7 is a cross-sectional layout diagram illustrating the configuration and operation of a light source of this embodiment that performs polarization conversion in a light source device 13 including a light guide 203 and LED elements 201, similar to Fig. 6. The light source device 13 is similarly composed of a light guide 203 formed of, for example, plastic, on the surface of which or inside which a light beam direction conversion means 204 is provided, an LED element 201 as a light source, a reflective sheet 205, a retardation plate 206, a lenticular lens, etc. A liquid crystal display panel 11 having polarizing plates on the light source light entrance surface and the image light exit surface is attached to the upper surface of the light source device 13.

[0160] Furthermore, a film- or sheet-like reflective polarizing plate 49 is provided on the light source light incidence surface (the lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, selectively reflecting one polarized wave (e.g., S wave) 211 of the natural light beam 210 emitted from the LED element 201. That is, in the example of FIG. 7 , the selective reflection characteristics of the reflective polarizing plate 49 are different from those in FIG. 7 . The reflected light is reflected by a reflective sheet 205 provided on one surface (the lower surface in the figure) of the light guide 203 and returns to the liquid crystal display panel 11. A retardation plate (λ / 4 plate) is provided between the reflective sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49. The reflected light (reflected light beam) is reflected by the reflective sheet 205 and passes through the retardation plate (λ / 4 plate) twice, converting it from S-polarized light to P-polarized light. This improves the utilization efficiency of the light source light as image light. The image light beam, whose light intensity has been modulated by the image signal in the liquid crystal display panel 11, is emitted as shown by the arrow 214 in Fig. 7 and enters the retroreflector 2. After being reflected by the retroreflector 2, a real image, that is, a floating image in space, can be obtained.

[0161] In the light source device 13 shown in Figures 6 and 7, in addition to the function of the polarizer provided on the light incident surface of the corresponding liquid crystal display panel 11, a reflective polarizer reflects the polarized light component on one side. Therefore, the theoretically obtainable contrast ratio is the reciprocal of the cross transmittance of the reflective polarizer multiplied by the reciprocal of the cross transmittance obtained by the two polarizers attached to the liquid crystal display panel 11. This results in high contrast performance. In fact, experiments have confirmed that the contrast performance of the displayed image is improved by more than 10 times. As a result, high-quality images comparable to those of self-luminous organic EL displays are obtained.

[0162] <Example 2 of Display Device> Figure 8 shows another example of the specific configuration of the display device 1. The light source device 13 of this display device 1 is configured by housing LEDs, a collimator, a composite diffusion block, a light guide, etc. in a case made of, for example, plastic, and a liquid crystal display panel 11 is attached to the upper surface of the light source device 13. Also, LED elements 201, which are semiconductor light sources, and an LED board 202, on which a control circuit for the LED elements 201 is mounted, are attached to one side of the case of the light source device 13, and a heat sink 103, which is a member for cooling heat generated by the LED elements 201 and the control circuit, is attached to the outer surface of the LED board 202.

[0163] The liquid crystal display panel frame attached to the top surface of the case of the light source device 13 is configured to have attached thereto the liquid crystal display panel 11 attached to the frame, and further to have attached thereto an FPC 403 electrically connected to the liquid crystal display panel 11. That is, the liquid crystal display panel 11, which is the image display element 11, generates a display image together with the LED elements 201, which are solid-state light sources, by modulating the intensity of transmitted light based on a control signal from a control circuit (not shown) that constitutes the electronic device.

[0164] <Display Device Example 3> Next, another example of the specific configuration of the display device 1 (Display Device Example 3) will be described with reference to Fig. 9 . The light source device of this display device 1 converts a divergent beam of light (a mixture of P-polarized and S-polarized light) from the LED 201 into a substantially parallel beam by a collimator (LED collimator) 18, and reflects the parallel beam toward the liquid crystal display panel 11 by the reflective surface of the reflective light guide 304. The reflected light is incident on a reflective polarizer 49 disposed between the liquid crystal display panel 11 and the reflective light guide 304. The reflective polarizer 49 transmits light of a specific polarization (e.g., P-polarized light) and causes the transmitted polarized light to be incident on the liquid crystal display panel 11. Here, light polarized other than the specific polarization (e.g., S-polarized light) is reflected by the reflective polarizer 49 and directed again toward the reflective light guide 304.

[0165] The reflective polarizing plate 49 is installed at an angle with respect to the liquid crystal display panel 11 so that the reflective polarizing plate 49 is not perpendicular to the chief ray of light from the reflective surface of the reflective light guide 304. The chief ray of light reflected by the reflective polarizing plate 49 is incident on the transmission surface of the reflective light guide 304. The light that has entered the transmission surface of the reflective light guide 304 passes through the back surface of the reflective light guide 304, passes through the λ / 4 plate 270 which is a retardation plate, and is reflected by the reflector 271. The light reflected by the reflector 271 passes through the λ / 4 plate 270 again, and passes through the transmission surface of the reflective light guide 304. The light that has passed through the transmission surface of the reflective light guide 304 is incident on the reflective polarizing plate 49 again.

[0166] At this time, the light that re-enters the reflective polarizing plate 49 has passed through the λ / 4 plate 270 twice, and therefore its polarization has been converted to a polarization (for example, P-polarized light) that is transmitted through the reflective polarizing plate 49. Therefore, the light whose polarization has been converted passes through the reflective polarizing plate 49 and enters the liquid crystal display panel 11. Note that, with regard to the polarization design related to the polarization conversion, the polarization may be configured in reverse from the above explanation (S-polarized light and P-polarized light may be reversed).

[0167] As a result, the light from the LEDs 201 is aligned to a specific polarization (for example, P polarization), enters the liquid crystal display panel 11, and is brightness-modulated in accordance with the video signal to display an image on the panel surface. As in the above example, the light source has a plurality of LEDs 201, which are attached to predetermined positions relative to the corresponding collimators 18 of the plurality of collimators 18. However, since Figure 9 is a vertical cross section, only one LED 201 and one collimator 18 are shown.

[0168] Each collimator 18 is formed of a translucent resin such as acrylic or glass. The collimator 18 may have a cone-shaped outer circumferential surface obtained by rotating a parabolic cross section. The collimator 18 may have a concave portion with a convex portion (i.e., a convex lens surface) formed in the center of the apex (the side facing the LED substrate 202). The collimator 18 may have a convex lens surface (or a concave lens surface) protruding outward in the center of the flat portion (the side opposite the apex). The parabolic surface forming the cone-shaped outer circumferential surface of the collimator 18 is set within an angle range that allows total reflection of the light emitted from the LED 201 toward the periphery, or a reflective surface is formed therein.

[0169] The LEDs 201 are arranged at predetermined positions on the surface of the circuit board, that is, the LED board 202. The LED board 202 is arranged and fixed to the collimator 18 so that the LEDs 201 on the surface are positioned at the center of the apex of the conical convex shape (or in the concave portion if the apex has a concave portion).

[0170] With this configuration, the collimator 18 focuses the light emitted from the LED 201, particularly the light emitted from the central portion, into parallel light by the convex lens surface that forms the outer shape of the collimator 18. Light emitted from other portions toward the periphery is reflected by the parabolic surface that forms the outer peripheral surface of the conical shape of the collimator 18, and is similarly focused into parallel light. In other words, the collimator 18, which has a convex lens in its center and a parabolic surface formed on its periphery, makes it possible to extract almost all of the light generated by the LED 201 as parallel light, thereby improving the utilization efficiency of the generated light.

[0171] Furthermore, the light converted into approximately parallel light by the collimator 18 shown in FIG. 9 is reflected by the reflective light guide 304. Of this light, light of a specific polarization passes through the reflective polarizer 49 due to the action of the reflective polarizer 49, while light of the other polarization reflected by the action of the reflective polarizer 49 passes through the light guide 304 again. The light is reflected by the reflector 271 located opposite the liquid crystal display panel 11 with respect to the reflective light guide 304. At this time, the light is polarized and converted by passing twice through the λ / 4 plate 270, which is a retardation plate. The light reflected by the reflector 271 passes through the light guide 304 again and enters the reflective polarizer 49 provided on the opposite surface. Since the incident light has been polarized and converted, it passes through the reflective polarizer 49 and enters the liquid crystal display panel 11 with its polarization direction aligned. As a result, all of the light from the light source can be used, doubling the geometrical optical utilization efficiency of light. Furthermore, since the degree of polarization (extinction ratio) of the reflective polarizer 49 is also included in the extinction ratio of the entire system, the use of the light source device of this embodiment significantly improves the contrast ratio of the entire display device. Adjusting the surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflector 271 allows adjustment of the angle of light reflection and diffusion at each reflective surface. The surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflector 271 can be adjusted for each design to optimize the uniformity of the light incident on the liquid crystal display panel 11.

[0172] It should be noted that the λ / 4 plate 270, which is the retardation plate in Fig. 9, does not necessarily have to have a phase difference of λ / 4 with respect to polarized light that is perpendicularly incident on the λ / 4 plate 270. In the configuration of Fig. 9, any retardation plate may be used as long as the phase changes by 90° (λ / 2) when polarized light passes through it twice. The thickness of the retardation plate may be adjusted according to the incident angle distribution of the polarized light.

[0173] <Display Device Example 4> Another example (Display Device Example 4) of the configuration of the optical system, such as the light source device of the display device 1, will be described with reference to Fig. 10 . Display Device Example 4 is a configuration example in which a diffusion sheet is used instead of the reflective light guide 304 in the light source device of Display Device Example 3. Specifically, two optical sheets (in other words, diffusion sheets) that convert the diffusion characteristics in the vertical and horizontal directions of the drawing (front-to-back directions, not shown) are used on the light emission side of the collimator 18. The two optical sheets are shown as optical sheet 207A and optical sheet 207B. Light from the collimator 18 is incident between the two optical sheets.

[0174] The optical sheet may be a single sheet instead of a two-sheet configuration. In a single-sheet configuration, the vertical and horizontal diffusion characteristics are adjusted by the fine shapes of the front and back surfaces of the single optical sheet. Alternatively, multiple diffusion sheets may be used to share the functions. In the example of FIG. 10 , the reflection and diffusion characteristics due to the front and back shapes of optical sheets 207A and 207B can be optimally designed using the number of LEDs 201, the divergence angle from LED substrate 202, and the optical specifications of collimator 18 as design parameters so that the surface density of the light beam emitted from the liquid crystal display panel 11 is uniform. In other words, in the example of FIG. 10 , the diffusion characteristics are adjusted by the surface shapes of multiple diffusion sheets instead of light guides.

[0175] In the example of FIG. 10 , polarization conversion is performed in the same manner as in the display device example 3 described above. That is, in the example of FIG. 10 , the reflective polarizing plate 49 may be configured to have the property of reflecting S-polarized light (transmitting P-polarized light). In this case, the reflective polarizing plate 49 transmits P-polarized light from the light source LED 201, and the transmitted light enters the liquid crystal display panel 11. The reflective polarizing plate 49 reflects S-polarized light from the light source LED 201, and the reflected light passes through the retardation plate 270 shown in FIG. 10 . The light that passes through the retardation plate 270 is reflected by the reflector 271. The light reflected by the reflector 271 passes through the retardation plate 270 again and is converted to P-polarized light. The polarization-converted light passes through the reflective polarizing plate 49 and enters the liquid crystal display panel 11.

[0176] It should be noted that the λ / 4 plate 270, which is the retarder in FIG. 10, does not necessarily have to have a phase difference of λ / 4 with respect to polarized light that is perpendicularly incident on the λ / 4 plate 270. In the configuration of FIG. 10, any retarder that changes the phase by 90° (λ / 2) when polarized light passes through it twice may be used. The thickness of the retarder may be adjusted according to the distribution of incident angles of the polarized light. It should be noted that, in FIG. 10 as well, the polarization design for polarization conversion may be configured in reverse (reversing the S-polarized light and P-polarized light) from the above explanation.

[0177] In a typical TV device, the light emitted from the liquid crystal display panel 11 has similar diffusion characteristics in both the horizontal direction of the screen (shown by the X-axis in FIG. 12( a)) and the vertical direction of the screen (shown by the Y-axis in FIG. 12( b)). In contrast, the diffusion characteristics of the light beam emitted from the liquid crystal display panel 11 of this embodiment are, for example, as shown in Example 1 of FIG. 12 , when the viewing angle at which the luminance is 50% of that at a front view (0-degree angle) is set to 13 degrees, this is 1 / 5 of the 62-degree viewing angle of a typical TV device. Similarly, the vertical viewing angle is set asymmetrically between the top and bottom, and the reflection angle and the area of ​​the reflective surface of the reflective light guide are optimized to keep the upper viewing angle to about 1 / 3 of the lower viewing angle. As a result, the amount of image light directed toward the monitoring direction is significantly improved compared to conventional LCD TVs, with brightness being 50 times higher or more.

[0178] Furthermore, assuming the viewing angle characteristics shown in Example 2 of Figure 12, if the viewing angle at which brightness is 50% of that at a front view (angle of 0 degrees) is set to 5 degrees, this is 1 / 12 of the 62 degrees of devices used for general TV applications. Similarly, the vertical viewing angle is set equal at the top and bottom, and the reflection angle and area of ​​the reflective surface of the reflective light guide are optimized to keep the viewing angle to about 1 / 12 of that of devices used for general TV applications. As a result, the amount of image light directed in the monitoring direction is significantly improved compared to conventional LCD TVs, and brightness is more than 100 times higher.

[0179] As described above, by setting the viewing angle to a narrow angle, the amount of luminous flux directed in the monitoring direction can be concentrated, significantly improving the efficiency of light utilization. As a result, even when using a liquid crystal display panel for general TV applications, by controlling the light diffusion characteristics of the light source device, it is possible to achieve a significant improvement in brightness with similar power consumption, making it possible to create a video display device that is compatible with information display systems aimed at bright outdoor areas.

[0180] When using a large LCD panel, the brightness of the screen can be improved by directing the light from the periphery of the screen inward so that it is directed toward the observer when the observer is facing the center of the screen. Figure 11 shows the convergence angle between the long and short sides of the panel when the observer's distance from the panel, L, and the panel size (screen ratio 16:10) are used as parameters. When monitoring with the screen in portrait orientation, the convergence angle can be set to match the short side. For example, when using a 22-inch panel in portrait orientation and the monitoring distance is 0.8 m, a convergence angle of 10 degrees will allow the image light from the four corners of the screen to be effectively directed toward the observer.

[0181] Similarly, when monitoring with a 15-inch panel in portrait orientation and a monitoring distance of 0.8 m, a convergence angle of 7 degrees will allow the image light from the four corners of the screen to be effectively directed towards the monitor. As described above, depending on the size of the liquid crystal display panel and whether it is used portrait or landscape, the overall brightness of the screen can be improved by directing the image light from the periphery of the screen towards the monitor who is in the optimum position to monitor the centre of the screen.

[0182] As shown in Figure 9, the basic configuration involves a light source device directing a light beam with a narrow angle of directionality to a liquid crystal display panel 11, which is then luminance-modulated according to a video signal. The video information displayed on the screen of the liquid crystal display panel 11 is then reflected by a retroreflector, and the resulting spatially floating image is displayed indoors or outdoors via a transparent member 100.

[0183] By using the display device and light source device according to the embodiment of the present invention described above, it is possible to realize a space floating image display device with higher light utilization efficiency.

[0184] <Example of Image Display Processing in Space-Floating Image Display Device> Next, an example of a problem solved by the image processing of this embodiment will be described with reference to Fig. 13A. In the space-floating image display device 1000 (Fig. 3, Fig. 4A to Fig. 4P), when the far side of the space-floating image 3 from the user's perspective is inside the housing of the space-floating image display device 1000 and it is sufficiently dark, the user will visually recognize that the background of the space-floating image 3 is black.

[0185] Here, an example of displaying a character "panda" 1525 in a floating-in-space image 3 will be described with reference to Fig. 13A. First, for an image including a pixel region for drawing the image of the character "panda" 1525 and a transparent information region 1520 that is a background image, as shown in Fig. 13A(1), the image control unit 1160 in Fig. 3 distinguishes and recognizes the pixel region for drawing the image of the character "panda" 1525 from the transparent information region 1520 that is a background image.

[0186] A method for distinguishing and recognizing the character image from the background image is, for example, to configure the image processing of the video control unit 1160 so that the background image layer and the layer of the character image in front of the background image layer can be processed as separate layers, and the character image and background image can be distinguished and recognized based on the superimposition relationship when these layers are combined.

[0187] Here, the image control unit 1160 recognizes black pixels that depict objects such as character images and transparent information pixels as different information. However, it is assumed that both the black pixels that depict objects and the transparent information pixels have a luminance of 0. In this case, when the Space Floating Image 3 is displayed, there is no difference in luminance between the pixels that depict black in the image of the character "panda" 1525 and the pixels of the transparent information region 1520, which is the background image. Therefore, in the Space Floating Image 3, as shown in FIG. 13A (2), neither the pixels that depict black in the image of the character "panda" 1525 nor the pixels of the transparent information region 1520 have luminance, and they are visually perceived by the user as the same optically black space. In other words, the black portions of the image of the character "panda" 1525, which is an object, blend into the background, and only the non-black portions of the character "panda" 1525 are perceived as floating in the display region of the Space Floating Image 3.

[0188] An example of image processing according to this embodiment will be described with reference to FIG. 13B. FIG. 13B is a diagram illustrating an example of image processing that more effectively resolves the issue of the black image region of the object blending into the background, as described in FIG. 13A. In FIGS. 13B(1) and 13B(2), the upper side shows the display state of the floating-in-space image 3, and the lower side shows the input / output characteristics of the image processing of the image of the object. Note that the image of the object (character "panda" 1525) and the corresponding data may be read from the storage unit 1170 or memory 1109 in FIG. 3, or may be input from the video signal input unit 1131, or may be acquired via the communication unit 1132.

[0189] In the state shown in Figure 13B(1), the input / output characteristics of the image processing of the object image are in a linear state with no particular adjustment. In this case, the display state is the same as in Figure 13A(2), and the black image area of ​​the object blends into the background. In contrast, in Figure 13B(2), the video control unit 1160 of this embodiment adjusts the input / output characteristics of the image processing of the image of the object (character "panda" 1525) to the input / output characteristics shown in the lower row.

[0190] That is, the video control unit 1160 performs image processing with input / output characteristics on the image of the object (character "panda" 1525), which has a characteristic of converting pixels in low-brightness areas of the input image into output pixels with increased brightness values. After the image of the object (character "panda" 1525) has been subjected to the image processing with input / output characteristics, a video including the image of the object (character "panda" 1525) is input to the display device 1 and displayed. Then, as shown in the upper part of FIG. 13B (2), the display state of the floating in space image 3 is such that the brightness of pixel areas depicting black in the image of the character "panda" 1525 increases. This allows the user to distinguish the areas depicting black among the areas depicting the image of the character "panda" 1525 without them blending into the black background, making it possible to display the object more appropriately.

[0191] 13B(2), the area displaying the image of the object character "panda" 1525 can be distinguished from the black background inside the housing of the space-floating image display device 1000 through the window, improving the visibility of the object. Therefore, for example, even if the object includes pixels with a brightness value of 0 among the pixels constituting the object before the image processing (i.e., at the time when the image of the object and the corresponding data are read from the storage unit 1170 or memory 1109 in FIG. 3, or at the time when the image of the object is input from the video signal input unit 1131, or at the time when the data of the object is acquired via the communication unit 1132, etc.), the image processing of the input / output characteristics by the video control unit 1160 converts the object into an object with increased brightness values ​​of pixels in low-brightness areas, and then the object is displayed on the display device 1 and converted into a space-floating image 3 by the optical system of the space-floating image display device 1000.

[0192] That is, the pixels that make up the object after image processing of the input / output characteristics are converted to a state in which they do not include pixels with a brightness value of 0, and then they are displayed on the display device 1 and converted into a floating image 3 by the optical system of the floating image display device 1000.

[0193] In the image processing of Figure 13B(2), a method for applying image processing with the input / output characteristics of Figure 13B(2) only to the image area of ​​the object (character "panda" 1525) is, for example, to configure the image processing of the video control unit 1160 so that the background image layer and the layer of the character image in front of the background image layer can be processed as separate layers, and the image processing with the input / output characteristics of Figure 13B(2) is applied to the character image layer while not applying this image processing to the background image layer.

[0194] Then, by combining these layers, image processing with a characteristic of raising the low-brightness areas of the input image is performed only on the character image, as shown in Fig. 13B (2). Alternatively, after the character image layer and background image layer are combined, image processing with the input / output characteristics of Fig. 13B (2) may be performed only on the character image area.

[0195] Furthermore, the input / output image characteristics used in the image processing for enhancing low-luminance regions of the input / output characteristics for the input image are not limited to the example shown in FIG. 13B(2). Any image processing for enhancing low luminance may be used, including so-called brightness adjustment. Alternatively, image processing for improving visibility by controlling the gain that changes the weighting of Retinex processing, as disclosed in International Publication WO 2014 / 162533, may be performed.

[0196] According to the image processing of FIG. 13B(2) described above, it is possible to make the user aware of areas where black is drawn among areas where images of characters, objects, etc. are drawn without blending into the black background, thereby realizing a more suitable display.

[0197] 13A and 13B, the problems and more suitable image processing were explained using examples of a space-floating image display device in which the background appears black (for example, the space-floating image display device 1000 in FIGS. 4A to 4G, or the space-floating image display device 1000 in a state in which the rear side window is shaded in FIGS. 4I and 4J). However, this image processing is also effective for devices other than these space-floating image display devices.

[0198] Specifically, in the space-floating image display device 1000 of Fig. 4H and the space-floating image display device 1000 in Fig. 4I and Fig. 4J where the rear window is not shaded, the background of the space-floating image 3 is not black, but the scenery behind the space-floating image display device 1000 through the window. In this case, the problems explained in Fig. 13A and Fig. 13B also exist.

[0199] That is, the part of the image of the character "panda" 1525, which is an object, that is drawn in black will blend into the scenery behind the space-floating image display device 1000 through the window. In this case, by using the image processing of Fig. 13B (2), the part of the image of the character "panda" 1525, which is an object, that is drawn in black will be recognized as being distinct from the scenery behind the space-floating image display device 1000 through the window, improving the visibility of the object.

[0200] That is, by using the image processing of FIG. 13B(2), the area displaying the image of the character "panda" 1525, which is an object, can be recognized as distinct from the scenery behind the space floating image display device 1000 through the window, and it can be more easily recognized that the character "panda" 1525, which is the object, is in front of the scenery, improving the visibility of the object.

[0201] 4K, 4L, and 4M, as described above, when another image (such as an image from the transmissive self-luminous image display device 1650 or an image from the second display device 1680) is displayed at a position different in depth from the space-floating image 3, the background of the space-floating image 3 is not black, but the other image. In this case, the problems described in FIGS. 13A and 13B also exist.

[0202] That is, the part of the image of the character "panda" 1525, which is an object, that is drawn in black will blend into the other image that is displayed at a different depth from the floating image in space 3. In this case as well, by using the image processing of Fig. 13B(2), the part of the image of the character "panda" 1525, which is an object, that is drawn in black will be recognizable as being distinct from the other image, improving the visibility of the object.

[0203] In other words, by using the image processing of Figure 13B (2), the area displaying the image of the object character "panda" 1525 can be recognized as distinct from the other image, and it can be more easily recognized that the object character "panda" 1525 is in front of the other image, improving the visibility of the object.

[0204] An example of the image display process of this embodiment will be described with reference to Fig. 13C. Fig. 13C shows an example of the image display process of this embodiment, in which the space floating image 3 and a second image 2050, which is another image, are simultaneously displayed. The second image 2050 may correspond to the image displayed by the transmissive self-luminous image display device 1650 of Fig. 4K or 4L. The second image 2050 may also correspond to the image displayed by the second display device 1680 of Fig. 4M.

[0205] That is, the example of the image display in Fig. 13C shows a specific example of the image display examples of the space-floating image display device 1000 in Figs. 4K, 4L, and 4M. In the example of this figure, a bear character is displayed in space-floating image 3. Areas other than the bear character in space-floating image 3 are displayed in black, and become transparent as a space-floating image. In addition, the second image 2050 is a background image in which a plain, a mountain, and a sun are drawn.

[0206] 13C , the floating-in-space image 3 and the second image 2050 are displayed at different depth positions. When the user 230 views the two images, the floating-in-space image 3 and the second image 2050, in the line of sight of the arrow 2040, the user 230 can view the two images superimposed on each other. Specifically, the bear character of the floating-in-space image 3 appears superimposed in front of the background of plains, mountains, and the sun depicted in the second image 2050.

[0207] Here, since the space floating image 3 is formed as a real image in the air, when the user 230 moves their viewpoint slightly, they can recognize the depth of the space floating image 3 and the second image 2050 due to parallax. Therefore, the user 230 can get a stronger sense of floating in space from the space floating image 3 while viewing the two images in an overlapping state.

[0208] An example of the image display process of this embodiment will be described with reference to Fig. 13D. Fig. 13D(1) is a diagram of the floating in space image 3, from the example of image display of this embodiment in Fig. 13C, as seen from the line of sight of the user 230. Here, a bear character is displayed in the floating in space image 3. The area other than the bear character in the floating in space image 3 is displayed in black, and the floating in space image is transparent.

[0209] 13D(2) is a diagram showing the second image 2050 in the example of the video display of this embodiment shown in FIG. 13C as viewed from the line of sight of the user 230. In the example shown in this figure, the second image 2050 is a background image in which a plain, a mountain, and a sun are depicted.

[0210] 13D(3) is a diagram showing the state in which the second image 2050 and the floating in space image 3 appear superimposed in the line of sight of the user 230, among the example of image display of this embodiment in Fig. 13C. Specifically, the bear character of the floating in space image 3 appears superimposed in front of the background of plains, mountains, and the sun depicted in the second image 2050.

[0211] Here, when the space-floating image 3 and the second image 2050 are displayed simultaneously, it is desirable to pay attention to the balance of brightness between the two images in order to more favorably ensure the visibility of the space-floating image 3. If the second image 2050 is too bright compared to the brightness of the space-floating image 3, the displayed image of the space-floating image 3 will be transparent, and the second image 2050, which is the background, will be strongly visible through it.

[0212] Therefore, the output of the light source of the spatially floating image 3 and the display image brightness of the display device 1, and the output of the light source of the display device displaying the second image 2050 and the display image brightness of the display device should be set so that at least the brightness per unit area of ​​the spatially floating image 3 at the display position of the spatially floating image 3 is greater than the brightness per unit area of ​​the image light that reaches the display position of the spatially floating image 3 from the second image 2050.

[0213] Note that, since this condition only needs to be satisfied when the space-floating image 3 and the second image 2050 are displayed simultaneously, when switching from the first display mode in which the space-floating image 3 is not displayed and only the second image 2050 is displayed to the second display mode in which the space-floating image 3 and the second image 2050 are displayed simultaneously, control may be performed to reduce the brightness of the second image 2050 by lowering the output of the light source of the display device that displays the second image 2050 and / or the display image brightness of the display device. These controls may be realized by the control unit 1110 in Fig. 3 controlling the display device 1 and the display device that displays the second image 2050 (the transmissive self-luminous image display device 1650 in Fig. 4K or 4L or the second display device 1680 in Fig. 4M).

[0214] Note that when switching from the above-described first display mode to the above-described second display mode, if control is performed to reduce the brightness of second image 2050, the brightness may be reduced uniformly across the entire screen of second image 2050. Alternatively, instead of reducing the brightness uniformly across the entire screen of second image 2050, the brightness reduction effect may be greatest in the portion where an object is displayed in space-floating image 3, and the brightness reduction effect may be gradually weakened around that portion. In other words, the visibility of space-floating image 3 can be sufficiently ensured by reducing the brightness of second image 2050 only in the portion where space-floating image 3 is visually recognized as being superimposed on second image 2050.

[0215] Here, since the space floating image 3 and the second image 2050 are displayed at positions with different depths, when the user 230 slightly changes their viewpoint, the parallax causes a change in the superimposed position of the space floating image 3 relative to the second image 2050. Therefore, when switching from the above-mentioned first display mode to the above-mentioned second display mode, if the brightness is to be reduced unevenly across the entire screen of the second image 2050, it is not desirable to reduce the brightness sharply based on the outline of the object displayed in the space floating image 3, and it is desirable to perform gradation processing of the brightness reduction effect, which changes the brightness reduction effect stepwise depending on the position as described above.

[0216] In addition, in the space floating image display device 1000 where the position of the object displayed in the space floating image 3 is approximately at the center of the space floating image 3, the position where the brightness reduction effect of the gradation processing of the brightness reduction effect is greatest can be set to the center position of the space floating image 3.

[0217] According to the image display process of this embodiment described above, the user 230 can view the space floating image 3 and the second image 2050 more favorably.

[0218] Note that when displaying the space-floating image 3, control may be performed so as not to display the second image 2050. Since the visibility of the space-floating image 3 is improved when the second image 2050 is not displayed, this control is suitable for the space-floating image display device 1000 and the like, which are used in applications where the user must be able to reliably view the space-floating image 3 when the space-floating image 3 is displayed.

[0219] <Example 2> As Example 2 of the present invention, an example of another configuration example of a space-floating image display device will be described. Note that the space-floating image display device according to this example is obtained by changing the optical system stored in the space-floating image display device described in Example 1 to the optical system shown in Fig. 14(1) or Fig. 14(2). In this example, differences from Example 1 will be described, and repeated explanations of the same configuration as Example 1 will be omitted. Note that in the following description of this example, the predetermined polarized light and the other polarized light are polarized waves whose phases differ by 90° from each other.

[0220] Fig. 14(1) shows an example of an optical system and optical path according to this embodiment. The optical system shown in Fig. 14(1) is configured such that the display device 1 is closer to the polarization separation member 101B in the optical system of Fig. 2C, making the entire optical system more compact. In Fig. 14(1), components that are assigned the same reference numerals as those in Fig. 2C will not be described in detail again.

[0221] 14(1), similar to Fig. 2C, image light of a predetermined polarized light (P-polarized light in the figure) emitted from display device 1 travels in a direction perpendicular to the image display surface of display device 1. Here, similar to Fig. 2C, polarization separation member 101B selectively transmits the predetermined polarized light (P-polarized light in the figure) emitted from display device 1 and reflects the other polarized light (S-polarized light in the figure).

[0222] Therefore, image light of a predetermined polarization (P-polarized light in the figure) traveling in the vertical direction from the image display surface of the display device 1 passes through the polarization separation member 101B and reaches the retroreflector 2 to which the λ / 4 plate 21 is attached. The image light that is retroreflected by the retroreflector 2 and travels again toward the polarization separation member 101B has passed through the λ / 4 plate 21 twice, and is converted from the predetermined polarization (P-polarized light in the figure) at the time of emission from the display device 1 to the other polarization (S-polarized light in the figure). The image light that travels again toward the polarization separation member 101B is the other polarization (S-polarized light in the figure), and is therefore reflected by the polarization separation member 101B toward the position where the user should be. The traveling direction of the image reflected by the polarization separation member 101B is determined based on the angle at which the polarization separation member 101B is disposed.

[0223] In the example of Figure 14 (1), the image light traveling toward the polarization separation member 101B is reflected at a right angle by the polarization separation member 101B and travels as shown in the figure. The image light reflected by the polarization separation member 101B forms a floating image 3A in space. The floating image 3A can be viewed by the user from the direction of arrow A.

[0224] Here, due to the characteristics of retroreflection by the retroreflector 2, the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 to reach the position where the space floating image 3A is formed. This relationship determines the position where the space floating image 3A is formed in the traveling direction of the image light reflected by the polarization separating member 101B.

[0225] In the example of FIG. 14(1), the display device 1, the polarization separating member 101B, and the retroreflector 2 are arranged closer than in the example of FIG. 2C. This allows the entire optical system to be configured more compactly. However, the amount by which the space-floating image 3A protrudes from the optical system of FIG. 14(1) is not very large. For example, as an indicator of the amount by which the space-floating image 3A protrudes from the optical system, the distance (L1 in the example of FIG. 14(1)) from the position where the central light beam of the image light is reflected by the polarization separating member 101B to the position where the image light forms the space-floating image 3A is shown in the figure.

[0226] 14(1), the characteristics of P-polarized light and S-polarized light may be interchanged. Specifically, the predetermined polarization of the image light emitted from the display device 1 may be S-polarized light, and the characteristics of P-polarized light and S-polarized light may be interchanged with respect to the reflection characteristics of the polarization separation member 101B. In this case, the P-polarized light and S-polarized light shown in the figure are both reversed, but the optical design, such as the optical path, can be realized in exactly the same way.

[0227] Next, Fig. 14(2) shows another example of an optical system and an optical path according to this embodiment. The optical system of Fig. 14(2) is a modified version of the optical system of Fig. 14(1) in order to increase the amount of the floating image projecting from the optical system while realizing the same compactness as the optical system of Fig. 14(1). In Fig. 14(2), detailed descriptions of components with the same reference numerals as Fig. 14(1) will be omitted.

[0228] 14(2), similar to FIG. 14(1), image light of a predetermined polarized light (P-polarized light in the figure) emitted from display device 1 travels in a direction perpendicular to the image display surface of display device 1. Here, the polarization characteristics of polarization separation member 101B are arranged 90 degrees differently from those in FIG. 14(1). Image light of a predetermined polarized light (P-polarized light in the figure) traveling in a direction perpendicular to the image display surface of display device 1 passes through polarization separation member 101B.

[0229] 14(1), the image light passing through the polarization separation member 101B is not faced with the retroreflector 2 having the λ / 4 plate 21 attached thereto, but with the specular reflector 4 having the λ / 4 plate 21B attached thereto. Here, the reflection at the specular reflector 4 is specular reflection (also called regular reflection), not retroreflection.

[0230] Therefore, the image light that has passed through polarization separation member 101B is specularly reflected by specular reflector 4 to which λ / 4 plate 21B is attached. The image light that has been specularly reflected by specular reflector 4 and travels again toward polarization separation member 101B has been converted from the predetermined polarization (P-polarized in the figure) at the time of emission from display device 1 to the other polarization (S-polarized in the figure) by having passed through λ / 4 plate 21 twice. The image light that has traveled again toward polarization separation member 101B is the other polarization (S-polarized in the figure), and is therefore reflected by polarization separation member 101B.

[0231] Here, because the orientation of the polarization separation member 101B in Figure 14(2) is different from that in Figure 14(1), the image light reflected by the polarization separation member 101B travels in the opposite direction from the position where the user should be. A retroreflector 2 with a λ / 4 plate 21C attached is disposed at the destination of the image light reflected by the polarization separation member 101B. The image light is retroreflected by the retroreflector 2. The image light that is retroreflected by the retroreflector 2 and travels again toward the polarization separation member 101B has been converted from the other polarized light (S-polarized light in the figure) to the specified polarized light (P-polarized light in the figure) by passing through the λ / 4 plate 21C twice.

[0232] The image light that travels back toward the polarization separation member 101B is of a predetermined polarization (P polarization in the figure), so it passes through the polarization separation member 101B and continues toward the location where the user should be. The image light that has passed through the polarization separation member 101B forms a floating image 3B in space. The floating image 3B can be easily viewed by the user from the direction of arrow A.

[0233] 14(2), as in Fig. 14(1), due to the characteristics of retroreflection by the retroreflector 2, the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 to reach the formation position of the space floating image 3B. This relationship determines the formation position of the space floating image 3B in the traveling direction of the image light transmitted through the polarization separation member 101B.

[0234] 14(2), the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 is longer than the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2 in FIG. 14(1). This is because in the optical system of FIG. 14(2), an optical path going back and forth between the polarization separation member 101B and the specular reflector 4, which does not exist in the optical system of FIG. 14(1), is added to the optical path length of the image light emitted from the display device 1 to reach the retroreflector 2.

[0235] As a result, in the optical system of Figure 14(2), the distance from the position where the central ray of the image light passes through the polarization separation member 101B to the position where the image light forms the space-floating image 3B (L2 in the example of Figure 14(2)) is significantly longer than the distance from the position where the central ray of the image light is reflected by the polarization separation member 101B to the position where the image light forms the space-floating image 3A (L1 in the example of Figure 14(1)) in the optical system of Figure 14(1).

[0236] 14(2), the characteristics of P-polarized light and S-polarized light may be interchanged. Specifically, the predetermined polarization of the image light emitted from the display device 1 may be S-polarized light, and the characteristics of P-polarized light and S-polarized light may be interchanged with respect to the reflection characteristics of the polarization separation member 101B. In this case, the P-polarized light and S-polarized light shown in the figure are both reversed, but the optical design, such as the optical path, can be realized in exactly the same way.

[0237] As described above, according to the optical systems of Fig. 14(1) and Fig. 14(2) in the second embodiment of the present invention, a more compact optical system can be realized. In particular, according to the optical system of Fig. 14(2), it is possible to increase the amount of the floating image projecting from the optical system, despite the more compact optical system.

[0238] In addition, when the optical system of Fig. 14(1) or Fig. 14(2) is incorporated into a space-floating image display device, it can be realized by replacing the optical system in the space-floating image display device described in Example 1 with the optical system of Fig. 14(1) or Fig. 14(2). Specifically, the optical system of Fig. 14(1) may be replaced with the optical system of the space-floating image display device of Fig. 4E, Fig. 4F, Fig. 4G, Fig. 4H, Fig. 4I, Fig. 4J, Fig. 4K, Fig. 4L, or Fig. 4M. In this case, since the optical system becomes compact, it is possible to make the housing of the space-floating image display device of each figure smaller.

[0239] Specifically, the optical system of Fig. 14(2) may be replaced with the optical system of the space-floating image display device of Fig. 4E, Fig. 4F, Fig. 4G, Fig. 4K, or Fig. 4L. In this case, it is possible to increase the amount of the space-floating image projecting from the optical system. Also, since the optical system becomes compact, it is possible to make the housing of the space-floating image display device of each figure smaller.

[0240] <Embodiment 3> As embodiment 3 of the present invention, an example of the configuration of a space floating image display device will be described. The space floating image display device according to this embodiment can similarly apply the configurations of each figure explained in embodiment 1 and embodiment 2 as a basic configuration. In this embodiment, differences from embodiments 1 and 2 will be mainly explained, and repeated explanations of the same configurations as embodiments 1 and 2 will be omitted.

[0241] The space-floating image display device of the third embodiment is provided as one component in a predetermined device such as a product vending machine. This space-floating image display device is provided as a user interface (UI) for operations and inputs such as product selection in the predetermined device. In other words, this space-floating image display device functions as an operation unit, an input unit, a display unit, and a UI unit. In the third embodiment, the space-floating image display device is applied to a product vending machine. However, it is not limited to this, and can be similarly applied to devices that require selection operations, such as public facilities.

[0242] [Issues, etc.] In devices such as product vending machines, it is thought that there are many users who do not want to directly touch shared objects that are touched by an unspecified number of people, such as physical buttons or display buttons on a touch panel. In other words, there is a demand for a non-contact operation function that allows users to operate objects such as buttons without contact. One of the features of the space floating image display device of each embodiment is that it provides such a non-contact operation function.

[0243] In the third embodiment, a UI using a space-floating image displayed by a space-floating image display device is applied to the operation unit of a product vending machine, thereby providing a contactless operation function that allows a user to select a product to purchase by a contactless operation.

[0244] Furthermore, when a floating image display device (in other words, an aerial display) is implemented in a device such as a vending machine, miniaturization to fit the device is required. Furthermore, when a floating image display device is implemented as an operation unit or UI in a device such as a vending machine, it is also required that the user's operation of the floating image display device, that is, the aerial operation of the floating image in the air, be simpler and easier to use. In Example 3, in order to meet these requirements, a floating image display device is mounted on the main body of the vending machine, and a suitable UI using floating images in the air is provided.

[0245] In Example 3, a vending machine equipped with a floating-in-space image display device implements a UI using floating-in-space images as an operation unit for predetermined operations such as product selection and confirmation. Operations such as product selection and confirmation are realized by accepting and detecting mid-air operations on the image of this UI. In Example 3, the image of this UI is a wheel-like image described below, which prompts the user to perform operations in vertical and horizontal directions (planar directions) corresponding to the two-dimensional plane of the floating-in-space image. In particular, Example 3 makes it possible to select one product selection button from multiple product selection buttons on the product display unit of the vending machine in response to mid-air operations on the image of this UI. Furthermore, in Example 3, control such as lighting up the selected product selection button is performed.

[0246] [Merchandise Vending Machine] Fig. 15 shows an example of the configuration of a merchandise vending machine 3000 equipped with the space-floating image display device 1000 of Example 3 as an operation unit 1510. The operation unit 1510 may be equipped with any of the space-floating image display devices 1000 described in Example 1 or Example 2. Hereinafter, in various descriptions of the merchandise vending machine of this embodiment, the space-floating image display device 1000 which is the operation unit 1510 has, for example, the configuration shown in Fig. 3. The operation of each component of the space-floating image display device 1000 which is the operation unit 1510 is controlled by the control unit 1110 as described in Fig. 3. The image display operation of the space-floating image display device 1000 which is the operation unit 1510 is displayed by the display device 1 based on the control of the image control unit 1160 as described in Fig. 3. As described in FIG. 3, the operation detection operation of the operation unit 1510, the airborne operation detection unit 1350, performs operation detection processing based on the sensing results of the airborne operation detection sensor 1351, and generates control information based on the detected operation. The functions and operations of the other components shown in FIG. 3 are the same as those described in Example 1 or Example 2, so repeated explanations will be omitted. FIG. 15 shows an x-z ​​plan view of the front surface 3591 of the housing 3590 of the main body 3500 of the product vending machine 3000 as viewed from the front by the user. The main body 3500 of the product vending machine 3000 has a roughly rectangular housing 3590. The front surface 3591 of the housing 3590 is provided with a product display unit 1501, product selection buttons 1502, and the like. The front surface 3591 is also provided with a product removal slot 3592, a coin / bill insertion slot 3593, a payment device 1702, and the like.

[0247] In this embodiment, on the front surface 3591, an operation unit 1510 of the space floating image display device 1000 is provided in an area to the right of the product display unit 1501, for example, above the payment device 1702. The position where this operation unit 1510 is placed is one example. In other words, the operation unit 1510 is an operation unit or an operation device.

[0248] In the operation unit 1510, a space-floating image 3 is formed and displayed by the space-floating image display device 1000 in a rectangular opening 1515 on the front surface 3591. The display range 3R of the space-floating image 3 is set, for example, to match the size of the opening 1515. This space-floating image 3 includes a UI image 1520 for accepting aerial operations by the user, i.e., the product purchaser. This image 1520, which will be described in detail later, is a wheel-like image that prompts planar operations (including operations in the x- and z-directions) of the space-floating image 3 on the x-z plane. In the example of FIG. 15 , this image 1520 is an image / image expressed as a three-dimensional sphere. In other words, this image 1520 is an operation image, a UI image, a wheel image, a sphere image, an object image, or the like.

[0249] In Example 3, the user looks at the image 1520 on the operation unit 1510 and performs an aerial operation, in other words, a touch operation, on the image 1520 with a finger 231 (for example, one finger). This aerial operation includes at least an operation in the illustrated x direction (in other words, the horizontal or left-right direction) and z direction (in other words, the vertical or up-down direction). For the purpose of explanation, two arrows shown in the image 1520 represent the operation in the x direction and the z direction and the rotation of the wheel image 1520 in response to the operation. The space-floating image display device 1000 detects the aerial operation on the image 1520. This detection includes detection of the movement of the touch / contact position of the finger 231 in the x direction and the z direction, in other words, the displacement.

[0250] The product vending machine 3000 selects one of the multiple product selection buttons 1502 on the product display unit 1501 based on information detected by the air-operated image display device 1000 of the operation unit 1510 on the image 1520, and performs control such as lighting up the selected product selection button 1502. This allows the user to be informed that the product associated with the lit product selection button 1502 has been selected for purchase.

[0251] [Product Vending Machine: Configuration Example] Fig. 16 shows a configuration example of a product vending machine 3000. The product vending machine 3000 has a product vending machine main body 3500 and an operation unit 1510 using the space floating image display device 1000. The product vending machine main body 3500 includes a control unit 3501, memory 3502, non-volatile memory 3503, a communication unit 3504, a communication interface 3505, data storage 3506, a position information acquisition unit 3507, a power supply 3508, an external power supply input interface 3509, and a product vending machine function unit 3510, which are interconnected by an architecture such as a bus.

[0252] The control unit 3501 includes a processor and functions as a controller. The control unit 3501 controls each unit and the entire product vending machine 3000. The memory 3502 stores various data and information processed by the control unit 3501. The non-volatile memory 3503 holds various data and information handled by the control unit 3501. The data storage 3506 stores various data and information such as programs and video data. The communication unit 3504 is equipped with, for example, a wireless communication interface and can communicate with external devices. The communication interface 3505 is a communication interface with the space-floating image display device 1000 and communicates with the space-floating image display device 1000. The location information acquisition unit 3507 is equipped with a GPS receiver or the like and can acquire location information. The power supply 3508 supplies the necessary power to each unit of the product vending machine 3000 based on external power input from the external power input interface 3509.

[0253] The space-floating image display device 1000, which is the operation unit 1510, communicates with the communication interface 3505 of the product vending machine main body 3500 via the communication unit 1132 under the control of the control unit 1110 in Fig. 3. Through this communication, various control information such as control information based on the detection of aerial operations can be sent from the space-floating image display device 1000 to the product vending machine main body 3500. Note that the external power input interface 1111 in Fig. 3 of the space-floating image display device 1000, which is the operation unit 1510, is connected to the power supply 3508 of the product vending machine main body 3500 in Fig. 16. This ensures a power source to operate the space-floating image display device 1000, which is the operation unit 1510.

[0254] [Merchandise Vending Machine Functional Unit] Fig. 17 shows an example of the configuration of the merchandise vending machine functional unit 3510 of the merchandise vending machine main body 3500. The merchandise vending machine functional unit 3510 is a component for realizing the functions of the merchandise vending machine 3000. Publicly known technology can be applied to the merchandise vending machine functional unit 3510. The merchandise vending machine functional unit 3510 includes a merchandise display unit / display 1501, a video processing unit 1701, an electronic information medium reader / writer 1702, an imaging unit 1703, an audio output unit 1704, an audio input unit 1705, a coin / bill processing unit 1706, a merchandise storage unit 1707, a merchandise vending mechanism 1708, and the like, which are interconnected by an architecture such as a bus.

[0255] The product display unit / display 1501 is a product display unit or display, and although the illustration shows both a product display unit and a display as one, either implementation is acceptable. When configured as a product display unit, actual products or models are displayed, and there is no display. When configured as a product display unit, lighting devices are also provided to illuminate the displayed products. When configured as a display, product images are displayed on the screen of the display. The display may be a touch panel. When configured as a product display unit, the product selection button 1502 is configured to have a physical operation button and a light-emitting unit. When controlling the lighting of the product selection button 1502, the light-emitting unit is controlled to emit light. When configured as a display, the product selection button 1502 may be displayed as a button image on the screen of a touch panel display, for example, or the product image itself may function as the button.

[0256] When the product display unit / display 1501 is configured as a display (e.g., a touch panel), the video processing unit 1701 processes video information / video data, creates data / signals for displaying the video on the display screen, and displays the video on the display screen based on the data / signals.

[0257] The electronic information medium reader / writer 1702 is a device that reads and writes data from and to an electronic information medium, and functions as a payment device 1702. The payment device 1702 reads and writes data, for example, from an IC card of the user 230 or from NFC (near field communication) of a smartphone. In this embodiment, the payment device 1702 using the electronic information medium reader / writer 1702 is provided as part of the product vending machine function unit 3510. In this embodiment, the control unit 3501 mainly performs the cooperation and control of the operation unit 1510 with the space-floating image display device 1000. However, without being limited to this, for example, the payment device 1702 may also perform the cooperation and control with the space-floating image display device 1000.

[0258] The imaging unit 1703 uses a camera to capture an image of the vicinity of the housing 3590 in Fig. 15 , for example, the vicinity of the front surface 3591. As a result, the imaging unit 1703 also functions as a human presence sensor that detects a user near the front surface 3591. In addition to the imaging unit 1703, various sensors such as an infrared sensor may be provided. The imaging unit 1703 may be realized as part of the payment device 1702. User authentication may be performed using the imaging unit 1703. One or both of the imaging unit 1703 of the product vending machine 3000 and the imaging unit 1180 (Fig. 3) of the space-floating image display device 1000 may be used, or may be integrated into one.

[0259] The audio output unit 1704 outputs a predetermined sound from a speaker. The audio input unit 1705 inputs the user's voice through a microphone. The coin / bill processing unit 1706 processes coins or bills inserted by the user through the coin / bill insertion unit 3593. The product storage unit 1707 stores products such as beverages. The product vending mechanism 1708 vends purchased products from the product storage unit 1707 to the product removal slot 3592.

[0260] The communication interface 3505 communicates with the space-floating image display device 1000 of the operation unit 1510. Control information based on the detection of an aerial operation from the space-floating image display device 1000, which is the operation unit 1510 (hereinafter referred to as detection information / control information), is transmitted from the space-floating image display device 1000, which is the operation unit 1510, to the control unit 3501 via this communication. In various explanations of the product vending machine 3000 of this embodiment, the user's aerial operation on the operation unit 1510 is reflected in the operation or processing of the product vending machine function unit 3510 via this communication and the control of the control unit 3501.

[0261] The control unit 3501 controls the product selection buttons 1502 of the product display unit 1501 based on cooperation via communication between the operation unit 1510 and the space floating image display device 1000. For example, the control unit 3501 transmits a control signal 1711 to the product selection buttons 1502 based on detection information / control information from the operation unit 1510. The signal 1711 is a signal for lighting up the product selection buttons 1502 that have been selected by a user's mid-air operation, for example, and is a signal that instructs turning the lighting on / off, etc. Based on the signal 1711, the light-emitting units of the selected product selection buttons 1502 are illuminated.

[0262] [Product Vending Machine: Mounting of Space-Floating Image Display Device] Figure 18 shows a side view of an example of mounting the space-floating image display device 1000 on the product vending machine main body 3500. This example is an example based on the optical system of Figure 2D. The configuration example of the space-floating image display device 1000 is basically the same as the configuration examples of Figures 4O and 4P, with the difference being that the overall orientation of the arrangement is rotated, and the emission direction of the image light that forms the space-floating image 3 is horizontal (-y direction).

[0263] Image light is emitted downward (-z direction) from the display device 1 arranged on the upper side within the housing 3590. The image light is retroreflected by a retroreflector 5 arranged diagonally within the housing 3590, and the reflected light is emitted forward (-y direction). The reflected light forms a real image, a floating image 3 in space, at an opening 1515 in the front surface 3591 of the housing 3590.

[0264] In the example of Figure 18, the direction of emission of the image light is horizontal (-y direction), and a spatial floating image 3 (display range 3R) is formed that is arranged vertically in accordance with the position of the front surface 3591, but this is not limited to this.

[0265] There is a limit to the length in the depth direction (y direction) of the housing 3590 of the product vending machine main body 3500. In this embodiment, the space-floating image display device 1000 mounted in the housing 3590 is a space-floating image display device 1000 that has been miniaturized to fit the limit on the length in the depth direction (y direction) of the housing 3590, and has a relatively small depth length. In accordance with the miniaturization, the display range 3R of the space-floating image 3 by this space-floating image display device 1000 becomes a relatively small screen. The wheel image 1520 is displayed within a screen of this size.

[0266] [Product Vending Machine: Product Display Unit] Figure 19 is a side view showing an example of the configuration of the product display unit 1501 in the product vending machine main body 3500. As an example of the operation of the product vending machine, when a user operates the product selection button 1502, the configuration and operation are as follows. The user 230 looks at the product display unit 1501 and selects the product selection button 1502 corresponding to the product to be purchased, that is, presses it as a direct contact operation. In response to pressing the product selection button 1502, the product selection button 1502 lights up. This illumination indicates that the product has been selected. The user 230 then operates, for example, the payment device 1702 or the coin / bill insertion unit 3593 to purchase or obtain the product.

[0267] The way in which the product selection buttons 1502 are lit may vary depending on the implementation details of the product vending machine 3000. Another example may be as follows: When the user 230 inserts coins / bills, the product selection buttons 1502 of all products that can be purchased within the amount of the inserted coins / bills are lit in a predetermined manner (hereinafter referred to as the first lighting). From these product selection buttons 1502, the user 230 selects, i.e., presses, one product selection button 1502 that corresponds to the product that the user decides to purchase. In response to pressing the product selection button 1502, the product is purchased. In response to pressing the product selection button 1502, the product selection button 1502 may be lit in a predetermined manner (hereinafter referred to as the second lighting).

[0268] In contrast, in the third embodiment, the following configuration and operation are possible, and direct contact with the product selection button 1502 by pressing the button is not required. The user 230 checks the product to be purchased by looking at the product display unit 1501. The user 230 selects the product selection button 1502 corresponding to the desired product by performing an aerial operation on the wheel image 1520 of the operation unit 1510. In response to this selection operation, the product vending machine 3000, particularly the control unit 3501, lights up the product selection button 1502. This illumination indicates a selected state. The user 230 then decides to purchase the product corresponding to the selected product selection button 1502, for example, by operating the decision button described below. The user 230 then purchases or acquires the product by, for example, operating the payment device 1702 or the coin / bill insertion unit 3593.

[0269] Another example may be as follows: When the user 230 inserts a coin / bill, the product selection buttons 1502 for all products that can be purchased within the inserted amount are illuminated in a predetermined manner (first illumination). The user 230 performs an aerial operation on the wheel image 1520 of the operation unit 1510 to select one product selection button 1502 that corresponds to the desired product to be purchased from the product selection buttons 1502 that are illuminated in the first illumination state. In response to this selection operation, the product vending machine 3000, particularly the control unit 3501, controls the product selection button 1502 to illuminate in a predetermined manner (second illumination). The second illumination indicates that the product has been selected. The user 230 then operates, for example, the payment device 1702 or the coin / bill insertion unit 3593 to purchase or acquire the product.

[0270] [Product Vending Machine: Operation Unit (1)] FIG. 20A shows a side view of an example configuration of the operation unit 1510 in Example 3. The example configuration of FIG. 20A corresponds to the example configuration of FIG. 18 . The housing 3590 has an opening 1515 recessed toward the rear (+y direction) of the front surface 3591. Based on the image light A1 emitted from the space-floating image display device 1000 traveling in the horizontal direction (-y direction), a space-floating image 3 is formed in the opening 1515 at a position aligned with the front surface 3591 in the y direction. In this example, the space-floating image 3 is positioned so that it stands vertically (z direction). The display range 3R of the space-floating image 3 forms the x-z plane. The user 230 can best view the space-floating image 3 in the line of sight (+y direction) indicated by arrow A, which corresponds to the direction of the image light A1.

[0271] 20A also shows a front view (x-z plane view) of the floating-in-space image 3 when viewed from the front. The user 230 looks at the wheel image 1520 in the floating-in-space image 3 and performs mid-air operations such as touching and sliding with the finger 231. This allows for product selection and purchase decision. Therefore, in this third embodiment, the user 230 does not need to directly touch, i.e., press, the product selection button 1502, which is a physical button on the product display unit 1501 as shown in FIG. 19.

[0272] Of course, depending on the user 230 and the situation, a touch operation of directly pressing the physical product selection button 1502 (FIG. 19) may also be acceptable. In this case, product selection and the like can be performed using this touch operation, as in the past. The mechanism for touch operation of physical buttons such as this product selection button 1502 can be realized in the same way as in the past, and coexists with the mechanism for mid-air operation of the floating-in-space image 3 in this embodiment. In this embodiment 3, a touch operation in which the user 230 directly presses the product selection button 1501 is also permitted.

[0273] Furthermore, depending on the user 230 and the situation, both the mid-air manipulation of the floating-in-space image 3 and the touch manipulation of a physical button such as the product selection button 1501 may be used in combination. For example, the user 230 first selects the product selection button 1502 of a certain product by manipulating the wheel image 1520 in the air. After that, when the user 230 looks at the product display section 1501 and changes to purchasing another product, the user 230 may directly touch the product selection button 1502 corresponding to that other product. This touch manipulation puts that other product into a selected state. In another example, the user 230 first puts a certain product into a selected state by directly touching the product selection button 1502 of that product. After that, when the user 230 changes to purchasing another product, the user 230 can select the product selection button 1502 of that other product by manipulating the wheel image 1520 in the air.

[0274] [Product Vending Machine: Operation Unit (2)] FIG. 20B shows a side view of an example configuration of the operation unit 1510 as a modified example. In the example of FIG. 20B, a frame 1530, which is a physical frame that forms the top, bottom, left, and right sides of the opening 1515 of the operation unit 1510, is arranged so that the opening 1515 protrudes slightly toward the user (in the -y direction) from the front surface 3591 of the housing 3590. The distance from the front surface 3591 to the floating-in-space image 3 is indicated by distance d20B. This frame 1530 is a portion that surrounds the top, bottom, left, and right sides of the display range 3R of the floating-in-space image 3. This frame 1530 has a width in the y direction that is equal to or greater than distance d20B. The floating-in-space image 3 is formed in the space within this frame 1530, at a position that is a distance d20B toward the user from the front surface 3591. The user 230 performs an aerial operation on the image 1520 of the floating image 3 so as to place his / her fingers 231 inside this frame 1530 .

[0275] [Product Vending Machine: Operation Unit (3)] Fig. 20C shows a side view of an example configuration of the operation unit 1510 as a modified example. In the example of Fig. 20C, the frame 1530 shown in Fig. 20B is not provided. The floating-in-space image 3 is formed at a position a distance d20C forward from the front surface 3591 of the housing 3590. As shown in the figure, a protective glass plate 1540, which is a transparent member, is provided in an area of ​​the front surface 3591 corresponding to the opening 1515. In addition to the protective glass plate 1540, other optical elements may be provided. The image light A1 emitted from the floating-in-space image display device 1000 passes through the protective glass plate 1540 to form the floating-in-space image 3 at the position shown in the figure.

[0276] [Product Vending Machine: Operation Unit (4)] Fig. 20D shows a side view of an example configuration of the operation unit 1510 as a modified example. The example of Fig. 20D is an example in which the space-floating image display device 1000 is mounted in the same manner as the example configuration of Fig. 4P. In the example of Fig. 20D, the image light A1 is emitted obliquely upward toward the front surface 3591 of the housing 3590, so that the optical axis of the space-floating image 3 is arranged to face obliquely upward. The image light A1 forms the space-floating image 3, which is arranged obliquely as shown in the figure, in the obliquely facing opening 1515. The user 230 can more easily view the space-floating image 3 in the line of sight indicated by the arrow B, which faces obliquely downward in correspondence with the optical axis of the space-floating image 3.

[0277] [Wheel Image] Figures 21 and 22 show display configuration examples for a wheel image 1520 in the space floating image 3 formed in the opening 1515 of the operation unit 1510. Either display configuration example may be adopted. The image 1520 in state A is expressed as a simple sphere as a three-dimensional object. This sphere is displayed in a predetermined color (for example, red). This sphere is also given a gloss and shadow to create a three-dimensional effect. Without being limited to this, the sphere in image 1520 may have multiple colors or patterns on the surface of the sphere.

[0278] In state B, in the display range 3R of the floating-in-space image 3, in addition to the spherical image 1520, a text image 2102 is additionally displayed. In this example, the text image 2102 is displayed separately above the spherical image 1520, but the text image 2102 may also be displayed superimposed on the sphere. The details of the text image 2102 are arbitrary, and may be an icon, animation, or the like. The text image 2102 may be a guide image such as "Please operate vertically and horizontally" or "Please select a product," for example.

[0279] In image 1520 in state C, lines 2103 resembling latitude and longitude are added to the surface of the sphere to make the state of rotation easier to understand. In response to the user's aerial operation, image 1520 of the sphere rotates, and the display state of lines 2103 resembling latitude and longitude changes in accordance with the state of rotation.

[0280] In the image 1520 in state D, an arrow image 2104 representing the rotation of the sphere is additionally displayed. The arrow image 2104 may be displayed in advance before the user performs an aerial manipulation. For example, the illustrated arrow image 2104 represents a manipulation in the x direction. Alternatively, the arrow image 2104 may not be displayed initially, but may be displayed only after the user performs an aerial manipulation. In this case, the arrow image 2104 represents the direction of the manipulation performed by the user. In addition to the arrow image, other effect images for representing a rotation manipulation, etc. may be displayed. In another example, the image 1520 may be displayed in a first color when no aerial manipulation is being performed, and in a second color when an aerial manipulation is being performed.

[0281] The background of the image 1520 (for example, a sphere) in the floating image in space 3 can be any display, and can be roughly black or white. It just needs to be configured so that the image 1520 stands out against the background. Also, although the display range 3R of the floating image in space 3 is a square, it is of course not limited to this.

[0282] 22, the image 1520 in state E is displayed as a three-dimensional object formed by combining a disk (first wheel) that can rotate in the x direction and a disk (second wheel) that can rotate in the z direction. The first wheel indicates that it can be rotated in the x direction, and the second wheel indicates that it can be rotated in the z direction. The objects are not limited to disks, and may be ring-shaped or the like.

[0283] The image 1520 in state F is displayed as a two-dimensional circle rather than a three-dimensional sphere, with up, down, left, and right arrow images superimposed on it. These arrow images indicate that operation is possible in the x and z directions. Alternatively, these arrow images may be displayed as four arrow images separated into four directions: up, down, left, and right.

[0284] The image 1520 in state G is displayed as a two-dimensional cross-shaped object. This cross shape indicates that operation is possible in the x and z directions. This cross-shaped image may also be divided into four images, each separated into four directional parts: up, down, left, and right. In the case of an image in state F or state G, if the user 230 operates one of the four parts in mid-air, this may be determined as an operation in the direction corresponding to that part.

[0285] In state H, image 1520 is displayed as an object consisting of only a disk (first wheel) that can rotate in the x direction. For example, if operation in only the x direction is required, image 1520 may be displayed in this manner to accept / prompt operation in only the x direction. Similarly, only a disk (second wheel) that can rotate in the z direction may be displayed.

[0286] As a variant, the image 1520 in state I is composed of numeric buttons. This image 1520 does not accept input in the x or z directions, but is a button object for specifying and inputting numbers corresponding to product numbers. These numeric buttons may be provided as multiple buttons corresponding one-to-one with the product selection buttons 1502, or as a general-purpose predetermined number, for example, ten buttons numbered 0 through 9. The illustration shows a case in which ten buttons numbered 0 through 9 are provided. The user 230 can select and specify the desired product number by selecting one of these numeric buttons and performing an aerial operation (touch operation) one or more times. For example, if the user 230 touches the "1" button and the "2" button in succession, the user can select and specify the product number 12. Furthermore, in this variant, the floating-in-space image 3 may further display an image representing the state of the number selected or specified by the aerial operation (touch operation) of the numeric button image 1520. Separate number cancel buttons and number confirm buttons may also be provided.

[0287] [Merchandise Vending Machine: Sequence] Fig. 23 shows an example of the configuration of a sequence relating to processing and operations between the user 230 and the space-floating image display device 1000, which is the control unit 3501 and operation unit 1510 of the product vending machine 3000, in the system of Example 3. The main control processing in the space-floating image display device 1000 is performed by an image processing unit such as the control unit 1110 or image control unit 1160 in Fig. 3. Step S1 is the initial state of the product vending machine 3000 system. In this example, in the initial state, the space-floating image 3 is not displayed on the operation unit 1510. The control unit 3501 instructs the space-floating image display device 1000 to hide the space-floating image 3. In a modified example, the space-floating image 3 may be displayed on the operation unit 1510 even in the initial state.

[0288] Step S2 is detection of the user 230 approaching the product vending machine 3000. The product vending machine 3000 detects this approach using, for example, the camera of the imaging unit 1703. This detection may be detection of any user operation, such as touching a physical button. Based on this detection, in step S3, the control unit 3501 instructs the space-floating image display device 1000, which is the operation unit 1510, to display the space-floating image 3.

[0289] Step S4 is the display of the space-floating image 3 by the space-floating image display device 1000. In step S4, the space-floating image display device 1000 displays the space-floating image 3 in an initial state. For example, as in state A of Fig. 28 described later, an image 1520 of a wheel or the like is displayed. This initial state display is a display that encourages the user 230 to operate the wheel image 1520 in the air.

[0290] Step S5 is an aerial operation by the user 230 on the wheel image 1520 of the floating-in-space image 3, in other words, a product selection operation, a selection operation of the product selection button 1502. In step S6, the floating-in-space image display device 1000 detects the aerial operation of step S5 using a user operation detection mechanism and generates detection information. Through this detection, the floating-in-space image display device 1000 grasps, for example, the amount of displacement, the direction of displacement, etc. as information representing the aerial operation (see FIG. 24 described below).

[0291] In step S7, the control unit 1110 of the space-floating image display device 1000 transmits control information / signals for coordination to the control unit 3501 in response to the detected mid-air operation. This control information / signal may be information representing the mid-air operation (e.g., displacement amount, etc.), or product selection button selection information by mapping (FIG. 32), which will be described later. Here, the control unit 1110 of the space-floating image display device 1000 determines which product selection button 1502 is selected by mapping from the information representing the mid-air operation (e.g., displacement amount, etc.), and generates product selection button selection information representing that selection state.

[0292] Furthermore, in step S8, the image processing unit of the space floating image display device 1000 updates the display state of the wheel image 1520 in the space floating image 3 in accordance with the aerial operation detected in step S6. This is, for example, a display in which the wheel rotates in accordance with the aerial operation.

[0293] Meanwhile, in step S9, the control unit 3501 of the product vending machine 3000 executes predetermined control based on the control information / signal received in step S7. In this case, the predetermined control is control to light up a selected product selection button 1502 among the multiple product selection buttons 1502 on the product display unit 1501. As shown in FIG. 17 above, the control unit 3501 transmits a signal 1711 to the product selection button 1502 for controlling the lighting to occur. Based on the signal 1711, the light-emitting unit constituting the product selection button 1502 emits light.

[0294] The process and operation (steps S5 to S9) of selecting the product selection button 1502 corresponding to the above-mentioned mid-air operation can be repeated as many times as desired by the user 230.

[0295] Step S10 is an operation by the user 230 to decide to purchase the selected product. This operation is an airborne operation of deciding on the floating in space image 3 (particularly the decision button 2901 in FIG. 29 described later), or a touch operation of the physical decision button. In the case of an airborne operation (touch operation) of the decision button 2901 on the floating in space image 3 as in FIG. 29 described later, the user 230 performs an airborne operation (touch operation) on the decision button 2901. In the case where a physical decision button is provided on the outside of the floating in space image 3, the user 230 performs a touch operation (presses) on the physical decision button.

[0296] When the decision button 2901 on the space-floating image 3 is applied, in step S11, the space-floating image display device 1000 uses a user operation detection mechanism to detect the mid-air operation of the decision button 2901 as the purchase decision operation of step S10. When a physical decision button is applied, the control unit 3501 of the product vending machine 3000 detects the touch operation of the physical decision button. After step S11, the space-floating image display device 1000 transmits control information / signal representing the decision to purchase the product to the control unit 3501 of the product vending machine 3000.

[0297] In step S12, the control unit 3501 of the product vending machine 3000 executes control corresponding to the decision to purchase the product. In step S13, the user 230 performs the operation of payment and product acquisition. Correspondingly, in step S14, the control unit 3501 of the product vending machine 3000 controls the payment and product vending.

[0298] [Air Operation Detection] FIG. 24 is an explanatory diagram of the air operation and detection by the user's 230 finger 231 with respect to the wheel image 1520 in the floating-in-space image 3 of the opening 1515 of the operation unit 1510. First, in state A, as an example of the wheel image 1520, for ease of understanding, a point on the spherical surface of the wheel is illustrated as point P, and latitude and longitude lines passing through point P are also illustrated. Such point P and latitude and longitude lines may be actually displayed in the image 1520. Assume that point P has position coordinates (x0, z0) on the x-z plane of the display range 3R of the floating-in-space image 3. The user 230 performs an air operation on this wheel image 1520 using the user's finger 231 (e.g., one finger) as an operation object. The position where the finger 231 touches the image 1520 may be anywhere on the surface of the sphere. In state A, it is assumed that the fingertip touches (makes contact with) point Q, which is slightly shifted from point P. The position coordinates of point Q on the xz plane of the floating image 3 in space are (x1, z1).

[0299] In the modified example, since it is sufficient to detect the amount of displacement, the position touched by the finger 231 on the xz plane of the display range 3R of the floating in space image 3 may be an area outside the image 1520.

[0300] State B shows the case where, from state A, the user 230 moves their fingertip from point Q, for example, sliding it roughly to the right (+x direction), while keeping it touching (contacting) the x-z plane of the floating in space image 3. The position of the fingertip after the movement is point R. It is assumed that the position coordinates of point R on the x-z plane of the floating in space image 3 are (x2, z2). It is assumed that this movement takes a certain time T.

[0301] If the displacement amount (in other words, the position coordinate difference) when the touch point moves due to this mid-air operation is (Δx, Δz), then (Δx, Δz) = (x2 - x1, z2 - z1). If the speed during this movement is (vx, vz), then (vx, vz) = (Δx / T, Δz / T). This displacement amount has at least the x and z directions as displacement directions, and it is also possible to grasp diagonal displacement directions as shown in the figure.

[0302] The floating-in-space image display device 1000 can detect mid-air operations on the wheel image 1520 as shown in FIG. 24, particularly the position coordinates, displacement amount, displacement direction, etc. of the touch point as detection results, using, for example, the mid-air operation detection sensor 1351 and mid-air operation detection unit 1350 in FIG. 3, or the imaging unit 1180 as a user operation detection mechanism.

[0303] Point R in state B is a state in which the fingertip is touching the xz plane of the floating image 3 in space, but this is not limiting, and it may be a state in which the fingertip is not touching, such as in a flick operation.

[0304] The floating-in-space image display device 1000 can execute predetermined control based on the amount and direction of displacement detected by the aerial operation detection sensor 1351, etc. An example of the predetermined control is to select one product selection button 1502 on the product display unit 1501 according to the amount and direction of displacement, and link it to the control unit 3501 (described later).

[0305] Furthermore, the image control unit performs control to update the display of the wheel image 1520 based on the detection result of the aerial operation. In other words, the display state of the wheel image 1502 is changed so that it reacts to the aerial operation by the user 230. This gives the user 230 the feeling that the wheel is reacting and moving in response to their own aerial operation.

[0306] State C is an example after the display state of the wheel image 1520 has been changed from states A and B according to the detected mid-air operation (such as the amount of displacement). This display change is determined according to the detected amount of displacement, etc. In other words, the display is controlled so that the greater the amount of displacement or speed of the slide operation, the greater the change in the rotation state, etc. of the wheel image 1520. In this example, on the x-z plane of the floating in space image 3, the reference position of the wheel image 1520 is fixed, while the spherical surface changes according to the amount of displacement, etc. Point P in state A has moved to point S (x3, z3) in state C.

[0307] The display change of the wheel image 1520 is not limited to the above example, and may be, for example, a change in color or brightness, or an effect display may be added. Furthermore, audio output may be used in addition to the display. For example, an operation sound may be output in response to an operation such as touching or sliding the wheel image 1520. For audio output, the audio output unit 1140 in FIG. 3 or the audio output unit 1704 in FIG. 17 may be used.

[0308] Furthermore, in the above example, the reference position of the wheel image 1520 is fixed, but as a modified example, the position of the wheel image 1520 may be varied in response to an aerial operation within the xz plane of the display range 3R of the floating-in-space image 3. For example, when the user 230 performs a slide operation to the right, the wheel image 1520 may also be moved to a position where it has been slid to the right.

[0309] [Sensor of operation unit: camera] The detailed configuration of the sensor for detecting aerial operations on the wheel image 1520 of the floating image 3 in the operation unit 1510 of the product vending machine 3000 is not limited, but for example, the following sensors can be applied.

[0310] As a first example sensor, a camera or a TOF sensor (TOF: Time Of Flight) of the imaging unit 1180 (FIG. 3) can be applied. FIG. 25 shows a side view of an example arrangement of a sensor 2501 when a camera or a TOF sensor is applied as the sensor of the first example. The sensor 2501 is placed in a position that can cover imaging of the x-z plane of the floating-in-space image 3 from the rear side. The sensor 2501 is placed at a position behind the floating-in-space image 3 (y direction) so that the optical axis of the imaging faces forward (-y direction). In this example, the optical axis from the sensor 2501 is placed so that it faces diagonally downward. The imaging range of the sensor 2501 includes the x-z plane of the floating-in-space image 3. Furthermore, the imaging range of the sensor 2501 extends to the front side of the front surface 3591 via the opening 1515, so that the body of the user 230 who is in front of the floating-in-space image 3 can also be imaged. The space floating image display device 1000 can determine the location on the x-z plane of the space floating image 3 that is touched by the finger 231 based on analysis processing from the image of the camera that is the sensor 2501. Processing such as image analysis may be performed by the circuitry of the imaging unit 1180 in FIG. 3, or may be performed by an image processing unit such as the aerial operation detection unit 1350, the control unit 1110, or the image control unit 1160.

[0311] If the sensor 2501 is a TOF sensor (in other words, a distance measurement sensor), the light emitted from the TOF sensor is reflected at the point on the x-z plane of the floating image 3 that the finger 231 is touching, and the TOF sensor receives the returning reflected light. The TOF sensor can calculate the distance from the time it takes for the emitted light to return as reflected light. Therefore, it is possible to determine the point on the x-z plane of the floating image 3 that the finger 231 is touching based on that distance.

[0312] [Sensor of Operation Unit: Aerial Operation Detection Sensor] Fig. 26 shows a side view of an example of the arrangement of an aerial operation detection sensor 1351, as a second example of a sensor that can be applied to the floating-in-space image 3 of the operation unit 1510. The image processing unit such as the aerial operation detection unit 1350, control unit 1110, or image control unit 1160 in Fig. 3 uses the detection signal of the aerial operation detection sensor 1351 to detect and determine the aerial operation with respect to the display range 3R of the floating-in-space image 3.

[0313] 26 shows an example of the arrangement of the aerial operation detection sensor 1351 in the opening 1515, where the aerial operation detection sensor 1351 is arranged facing downward at an upper position on the front surface 3591 of the housing 3590, in line with the depth direction (y direction) position of the floating in space image 3 in the opening 1515. The optical axis of the emitted light from the aerial operation detection sensor 1351 covers the xz plane of the floating in space image 3.

[0314] FIG. 27 shows an example of the configuration of the aerial operation detection sensor 1351 arranged above the floating-in-space image 3 in FIG. 26 in an x-z ​​plan view. The aerial operation detection sensor 1351 has multiple optical elements 1351c arranged in the x direction. The optical elements 1351c are pairs of a light-emitting element 1351a and a light-receiving element 1351b. The light-emitting element 1351a is composed of, for example, an infrared element. The light-emitting element 1351a emits light a1, for example, infrared light, downward in the z direction. If the light a1 is not blocked by an object, it passes through the display range 3R. If the light a1 is blocked by an object, it is reflected by the object and returns as reflected light a2. The reflected light a2 is received by the light-receiving element 1351b.

[0315] For example, if a contact point a3 made by a user's finger is within the x-z plane of the display range 3R, light a1 is reflected from the contact point a3 and returns as reflected light a2. The light receiving element 1351b at a certain x-direction position detects the reflected light a2. This allows the aerial operation detection unit 1350 to determine that the contact point a3 is located at that x-direction position. Furthermore, the distance can be calculated using the TOF method from the time it takes for light a1 to return as reflected light a2. For example, the distance a4 to the contact point a3 can be calculated. This also allows the position coordinates of the contact point a3 on the x-z plane of the display range 3R to be determined.

[0316] Not limited to this example, the aerial operation detection sensor 1351 may be arranged above, below, or to the left or right of the x-z plane of the display range 3 R. Furthermore, the aerial operation detection sensor 1351 may be arranged at a position shifted in the front-to-rear direction, i.e., the y direction, or multiple aerial operation detection sensors 1351 may be arranged at multiple positions in the front-to-rear direction.

[0317] As in the above example, detection of an aerial operation (at least an operation in the x- and z-directions) on the wheel image 1520 in the xz plane of the floating image 3 in space can be detected with sufficiently high accuracy by an aerial operation detection sensor 1351 of a planar sensor type such as that shown in FIG. 27 .

[0318] 28 and 29 show examples of mid-air operations on wheel image 1520 of operation unit 1510 and display control in response to the mid-air operations. State A in FIG. 28 shows how user 230 operates wheel image 1520 by mid-air operations such as touching or sliding with finger 231 on wheel image 1520 displayed on floating-in-space image 3 of operation unit 1510. In floating-in-space image 3, not only wheel image 1520 but also guide image 2801 such as "Turn to select product" may be displayed, and the guide content may be simultaneously output as audio.

[0319] State B shows an example of selection of product selection button 1502 in product display section 1501 on front surface 3591 of housing 3590. In response to operation in state A, the selection state of product selection button 1502 changes as shown in state B. The position of the lit product selection button 1502 is operated and controlled in response to the operation of wheel image 1520 in the air (for example, displacement in the x and z directions).

[0320] In this example, 18 examples of products, from product A to product R, are arranged in three rows and six columns as shown in the figure on the product display unit 1501. Corresponding to these products, 18 product selection buttons 1502 are arranged in three rows and six columns. For ease of identification, the product selection buttons 1502 may be referred to as product selection buttons A, B, C, etc.

[0321] In the example of state B, the initially selected product selection button 1502 (hereinafter referred to as the first product selection button) is product selection button 1502c corresponding to product C. Furthermore, the product selection button 1502 (hereinafter referred to as the second product selection button) selected by user 230 through an aerial manipulation of wheel image 1520 is product selection button 1502q corresponding to product Q. The aerial manipulation of wheel image 1520 in this case is a sliding operation that includes a sliding operation to a certain extent downward (-z direction) and a sliding operation to a certain extent rightward (+x direction). Note that this sliding operation may be two sequential sliding operations separated in the x direction and the z direction, or may be a sliding operation in a single diagonal direction (+x, -z). The selected product selection button 1502 changes depending on the amount of displacement of the detected aerial manipulation as shown in FIG. 24 .

[0322] In response to a selection operation on the wheel image 1520 as in state A, the product selection button 1502 is selected as in state B, and the selected product selection button 1502 is controlled to light up. The selection state of the product selection button 1502 is updated on the time axis as needed. The control unit 3501 cooperates with the space-floating image display device 1000 to control the product display unit 1501 to light up the product selection button 1502 in the selected state. In the example of state B, the state changes from the first product selection button (C) being lit up to the second product selection button (Q) being lit up.

[0323] Additionally, in response to this product selection operation, a path 2802 of product selection change is configured as shown in the figure. This path 2802 is a path connecting the first product selection button (C) to the selected second product selection button (Q), and also includes product selection buttons 1502 (I, O, P) that are passed through along the way. The control unit 3501 cooperates with the space-floating image display device 1000 to light up at least the selected product selection button 1502 (for example, product selection button 1502q), but is not limited to this, and may also control the lighting of the product selection buttons 1502 along the path 2802. Control examples will be described later.

[0324] In this example, the initially selected first product selection button is, for example, the product selection button 1502c, but this is not limiting. In the initial state, there are cases where no product selection button 1502 is selected, and cases where a product selection button 1502 in a predetermined default position is selected (described later).

[0325] State C of FIG. 29 is a continuation of state B of FIG. 28 . For example, when product selection button Q (1502q) for product Q is selected as in state B, the space-floating image display device 1000 switches and updates the display content of the space-floating image 3 in accordance with the selection. State C shows an example of the display after switching. The space-floating image display device 1000 displays a decision button 2901 as part of the image in the space-floating image 3. In this example, the decision button 2901 is displayed above the wheel image 1520, but this is not limiting. Furthermore, in addition to the decision button 2901, a guide such as "Please press the decision button once you have decided on the product" may also be output. Furthermore, while the decision button 2901 is displayed in this example, other display content, such as a purchase button or a payment button, may also be displayed in accordance with the flow of product purchases from the product vending machine 3000.

[0326] In the case where the user 230 has not yet decided on the product and continues to select the product again by pressing the decision button 2901 in state C, the wheel image 1520 can be operated again in the air, similar to state A. In this case, the space floating image display device 1000 will erase the decision button 2901 and perform control similar to state A and state B.

[0327] In a modified example, the decision button 2901 may be displayed from the beginning at all times within the xz plane of the space floating image 3. However, as will be described later, it is preferable to devise a way to make it difficult to operate the decision button 2901 by mistake.

[0328] When the user 230 decides to purchase the selected product, the user 230 performs an air operation, for example, a touch operation, with the finger 231 to press the decision button 2901 in state C, as shown in state D. The floating-in-space image display device 1000 detects the air operation and determines that the purchase of the selected product has been decided.

[0329] State E is an example of a display in the floating image 3 when transitioning to the payment or product vending phase after State D. In this example, the wheel image 1520 is erased, and a guide image 2902 such as "Please insert money" is displayed, and the same content may be output as audio at the same time. In another example, a guide such as "Please touch your payment card" related to the payment device 1702 ( FIG. 17 ) may be used. When transitioning from purchase confirmation to payment, the user 230 operates the coin / bill insertion unit 3593 or the payment device 1702, and the main unit 3500 performs payment processing. The main unit 3500 then vends the purchased product using the product vending mechanism 1708.

[0330] 28 and 29 , initially, the decision button 2901 is not displayed, and the wheel image 1520 is displayed, and an aerial operation is performed on the wheel image 1520. After the aerial operation is detected and the product selection button 1502 is selected, the decision button 2901 is displayed and a purchase decision operation is accepted. In another display control example, while the wheel image 1520 is being operated in the air, that is, during detection, the decision button 2901 is not displayed even if the product selection button 1502 is selected. Then, after the selection state is generated, a certain period of time or more is waited, and if no aerial operation continues for more than the certain period of time, the decision button 2901 may be displayed.

[0331] [Initial State] Fig. 30 shows an example of an initial state in which the product selection button 1502 is lit in a predetermined default position for product selection. In the example shown as initial state A, the default position is the product selection button A (1502a) for product A in the upper left of the product display section 1501. The control unit 3501 and the space floating image display device 1000 share setting information (see Fig. 32 described below) regarding the default position for selection of the product selection button 1502. In initial state A, the control unit 3501 controls the product selection button A (1502a) to be lit.

[0332] In another example, the default position may be the bottom left, top right, or bottom right button among the multiple product selection buttons 1502 in the product display section 1501, or any position on the top, bottom, left, or right side of the product display section 1501. In another example, the default position may be the button (e.g., product selection button L) that is closest to the wheel image 1520 on the operation unit 1510. When initial state A is applied, the default position is easy for the user 230 to understand. Note that maximum value 3011 is an example of the maximum amount of movement when considering movement from one product selection button 1502 to another product selection button 1502 in the product display section 1501.

[0333] In the example shown as initial state B, the default position is a position near the center of the area of ​​the product display section 1501, which in this example is product selection button I (1502i) for product I in the second row and third column from the left. In this example, there is no product selection button 1502 exactly in the center position (the intersection of the dashed dotted lines) of the multiple product selection buttons 1502 on the product display section 1501, so the product selection button I (1502i) on the left, which is the closest position to it, is set as the default position. The product selection button J (1502j) on the right may also be set as the default position. In initial state B, the control unit 3501 controls product selection button I (1502i) to light up.

[0334] When initial state B is applied, it is possible to minimize the maximum amount of movement when considering the amount of movement of a selected product selection button 1502 from the product selection button 1502 in the default position approximately in the center to each of the other product selection buttons 1502. Maximum value 3012 is the maximum amount of movement.

[0335] In the prior art, there is no concept of product selection using mid-air manipulation of the operation unit 1510 or wheel image 1520 using the floating image 3 in space, and therefore there is basically no concept of a default position for selection of the product selection button 1502 in the initial state as in the example above. In this third embodiment, the concept and function of product selection using mid-air manipulation of the operation unit 1510 or wheel image 1520 using the floating image 3 in space is newly introduced, so the default position for selection of the product selection button 1502 in the initial state as in the example above can be used for control.

[0336] [Path of change in product selection] In this embodiment 3, as in the example of state B in Fig. 28 , it is possible to control the path (in other words, the trajectory) of a change in product selection, that is, the path resulting from a change in the selection state of the product selection button 1502, to be expressed by lighting up the product selection button 1502. Fig. 31A shows an example of controlling the lighting up of this path.

[0337] Example 1 of path control is similar to state B in FIG. 28 . The first product selection button initially selected is product selection button C (1502c) for product C, and the second product selection button next selected by mid-air manipulation of the wheel image 1520 is product selection button Q (1502q) for product Q. The control unit 3501 changes the state from one in which the first product selection button (C) is lit to one in which the second product selection button (Q) is lit. In other words, at a first point in time, only the first product selection button (C) is lit, and at a subsequent second point in time, only the second product selection button (Q) is lit. In this example, the product selection buttons 1502 that are passed along the path 3101 configured in accordance with the mid-air manipulation are controlled not to light up. By seeing this lighting, the user 230 can recognize that the selected product (corresponding product selection button 1502) has changed from C to Q.

[0338] The difference between example 1 and example 2 of path control is that the product selection buttons 1502 (I, O, P) that are passed along the same path 3101 are also controlled to light up sequentially. That is, starting with a state in which product selection button C is lit, product selection buttons I, O, and P light up in order in response to aerial manipulation, and finally the selected product selection button Q lights up. The product selection buttons 1502 (I, O, P) that are passed along the path are temporarily selected, even if only for a short time, in response to aerial manipulation. By seeing the lights on the path 3101, the user 230 can more easily recognize that the selected product (corresponding product selection button 1502) has changed from C to Q.

[0339] Furthermore, as a variation of Example 2, the following may be used. In the processing of the space floating image display device 1000, it is assumed that the only two product selection buttons 1502 that are selected in response to mid-air operations are the first product selection button (C) and the second product selection button (Q), and that product selection buttons I, O, and P are not selected. In this case, the control unit 3501 creates a similar path 3101 from product selection button C to product selection button Q, and controls the path 3101 to sequentially light up the product selection buttons I, O, and P that are passed through along the way from product selection button C, and finally light up product selection button Q. In this case, the product selection buttons I, O, and P that are passed through along the way are not selected even temporarily, but are controlled to light up in order to more clearly show the path 3101 to the user 230.

[0340] Although not shown, in another example, it is possible to represent a path diagonal to the x and z directions. For example, it is possible to configure a path diagonal from product selection button C to product selection button Q. This path is, for example, a path moving in order from product selection button C, J, to Q.

[0341] Example 3 of path control is a case where the default position for product selection in the initial state is product A (product selection button A) (initial state A in FIG. 30 ). Path 3103 is a path connecting product selection button A to product selection button Q selected by mid-air operation. In this example, control is also performed so that product selection button A in the default position does not light up.

[0342] Example 4 of route control differs from Example 3 in that it controls to also light up the product selection button A at the default position in the initial state. When the default position is used as the starting point as in Examples 3 and 4, control to light up the product selection buttons 1502 that are passed through along the route can also be applied in the same way as in Example 2.

[0343] 31B shows another example of path control as a modified example. State A is an example in which the user 230 inserts coins / bills as described above, and the product selection buttons 1502 for all products that can be purchased within the amount inserted are illuminated in a predetermined state (first illumination). In this example, 12 products A to L in the first and second rows are available for purchase, and the corresponding product selection buttons 1502 (A to L) are in the first illumination state. Here, the first illumination is indicated by an oval.

[0344] State B illustrates a case in which the user 230 selects the product selection button 1502k for one product (e.g., product K) from the product selection buttons 1502 (A-L) in the first illumination state in State A by manipulating the wheel image 1520 in mid-air. The selected product selection button 1502k enters a predetermined illumination state (second illumination). Here, the second illumination is represented by a star shape. In this case, the product vending machine 3000 controls the illumination (second illumination) of the product selection buttons 1502 along the path from the product selection button 1502 before selection (in other words, before movement) to the product selection button 1502 after selection (in other words, after movement), similar to the example of path control illustrated in FIG. 31A . For example, a path 3105 is configured from product A to the product selection button 1502k for product K. This path 3105 is configured within the range of the first illumination.

[0345] [Interaction between the Control Unit of the Product Vending Machine and the Space-Floating Image Display Device] Figure 32 shows an example of a configuration related to the interaction between the control unit 3501 of the product vending machine 3000 and the space-floating image display device 1000. In particular, this figure shows an example of a configuration related to the control of the mapping (association) between the mid-air operation of the wheel image 1520 and the selection of the product selection button 1502. The product display unit 1501 has a system for arranging multiple product selection buttons 1502 depending on the implementation. For control and interaction, the control unit 3501 and the space-floating image display device 1000 understand this system. At least one of the control unit 3501 and the space-floating image display device 1000 performs mapping (association) between the mid-air operation and the selection of the product selection button 1502 based on this system.

[0346] The space floating image display device 1000 has an image processing unit 3201, a display unit 3202, an optical system 3203, a user operation detection mechanism 3204, etc. The image processing unit 3201 can be configured by the control unit 1110, the image control unit 1160, or the mid-air operation detection unit 1350, etc. in Fig. 3. The display unit 3202 corresponds to the display device 1, etc. in Fig. 3. The optical system 3203 corresponds to the retroreflection unit 1101, etc. in Fig. 3. The user operation detection mechanism 3204 corresponds to the imaging unit 1180 or the mid-air operation detection sensor 1351 in Fig. 3.

[0347] The image processing unit 3201 creates data / signals for displaying an image on the screen of the display unit 3202 based on image processing of the image information / image data, and drives and controls the display unit 3202. The display unit 3202 displays an image on a screen (for example, the screen of the aforementioned liquid crystal display panel 11) according to the data / signals. The image light emitted corresponding to the image on the screen of the display unit 3202 is adjusted through the optical system 3203, and forms the space-floating image 3, which is a real image, at a predetermined position. The user 230 performs an aerial operation as a user operation on the screen that displays the space-floating image 3. The user operation detection mechanism 3204 detects the aerial operation on the screen that displays the space-floating image 3. The image processing unit 3201 executes predetermined processing based on the information detected by the user operation detection mechanism 3204.

[0348] In Fig. 32, video processing unit 3201 cooperates with control unit 3501 of product vending machine 3000 through communication via communication unit 1132 in Fig. 3 and communication interface 3505 in Fig. 16. Control unit 3501 controls, for example, the illumination of product selection button 1502 on product display unit 1501 based on information from video processing unit 3201. Note that when product selection button 1502 is pressed by a touch operation on the product vending machine main body 3500 side, control unit 3501 may cooperate with video processing unit 3201 via communication to perform predetermined control.

[0349] 32, at least one of the floating-in-space image display device 1000 (particularly the image processing unit 3201) and the product vending machine main body 3500 (particularly the control unit 3501) has a function to perform mapping relating to the mid-air operation of the wheel image 1520 and the selection state of the product selection button 502. In this embodiment, the image processing unit 3201 is provided with a function 3211 to perform this mapping, but in a modified example, the control unit 3501 may be provided with a function 3212 to perform this mapping.

[0350] When the video processing unit 3201 has a mapping function 3211, the following occurs. The video processing unit 3201 selects a product selection button 1502 through mapping processing in response to the detection of an aerial operation on the wheel image 1520 in the floating-in-space image 3. Specifically, the video processing unit 3201 manages and stores, in product selection management information 3221, in other words, mapping information, information on the system configuration of the multiple product selection buttons 1502 on the product display unit 1501 and information indicating which product selection button 1502 is currently selected. The video processing unit 3201 can select a product selection button 1502 based on this product selection management information 3221 and detection information such as the displacement amount described above ( FIG. 24 ). The video processing unit 3201 then transmits product selection button selection information 3231 to the control unit 3501 as information indicating the selected product selection button 1502. The control unit 3501 controls the selected product selection button 1502 to light up based on the received product selection button selection information 3231 .

[0351] When the control unit 3501 has a mapping function 3212, the following occurs. The image processing unit 3201 transmits detection information 3232, such as the aforementioned displacement amount, to the control unit 3501 in response to detecting an aerial operation on the wheel image 1520 in the floating-in-space image 3. The control unit 3501 selects a product selection button 1502 through mapping processing based on the received detection information 3232. Specifically, the control unit 3501 manages and stores, in product selection management information 3222, in other words, mapping information, information on the organization of the multiple product selection buttons 1502 in the product display unit 1501 and information indicating which product selection button 1502 is currently selected. The control unit 3501 can select a product selection button 1502 based on this product selection management information 3222 and the detection information 3232. The control unit 3501 then controls the selected product selection button 1502 to light up.

[0352] Furthermore, the control unit 3501 may appropriately transmit information indicating which product selection button 1502 is currently selected to the video processing unit 3201.

[0353] The setting information regarding the mapping may include the following: A setting value that determines whether the selected product selection button 1502 on the product display unit 1501 is moved one unit up, down, left, or right (in other words, whether it is moved to an adjacent button) depending on the amount of displacement of the mid-air operation.

[0354] [Prevention of Misoperation of Confirm Button (1)] Fig. 33 shows an example of the configuration of the operation unit 1510 in a modified example of Example 3, in an x-z ​​plan view of the front surface 3591 of the housing 3590. The purchase decision operation / processing corresponding to the pressing of the Confirm button 2901 described above is important, and it is desirable to prevent misoperation. Therefore, in this modified example, in order to prevent misoperation, a partition 3301, which is a physical frame, is provided near the Confirm button 2901. This partition 3301 is an object that separates the space to protect the Confirm button 2901 and can rest the finger 231.

[0355] In this example, a partition 3301 is provided in the opening 1515 in which the floating-in-space image 3 is placed, corresponding to the position where the above-mentioned enter button 2901 is displayed. This partition 3301 is provided in a shape that surrounds the enter button 2901 on the top, bottom, left, and right sides in the x-z plane. This partition 3301 is physically connected to the edge of the opening 1515, which is part of the housing 3590. Note that the shape of the partition 3301 is not limited to this, and it is possible to provide the partition 3301 at least in a position between the wheel image 1520 and the image of the enter button 2901.

[0356] A side view is also shown below. As shown, the partition 3301 has a width in the depth direction (y direction). The floating-in-space image 3 is arranged within this width. The bottom edge of the partition 3301 is located between the image of the enter button 2901 and the image of the wheel 1520. If the user 230 is performing an aerial operation such as a slide on the image of the wheel 1520 with his / her finger 231 and the finger 231 unintentionally shifts and moves toward the enter button 2901, the finger 231 will hit the partition 3301 and be stopped, as shown in the figure. This prevents an erroneous touch operation (pressing) the enter button 2901. When user 230 intends to touch (press) decision button 2901, he or she first moves finger 231 slightly toward the front (in the -y direction) to avoid partition 3301, and then, as shown in the figure, moves finger 231 again into the space within partition 3301 at the back, thereby touching (pressing) decision button 2901.

[0357] In this example, the decision button 2901 is configured with the space floating image 3, so it is easy to change the display to other object images. In this example, the decision button 2901 is shown as an example of an important object image for which erroneous operation should be prevented, but this is not limiting, and the configuration of the divider 3301 etc. can be similarly applied to other important object images.

[0358] A general characteristic of a space-floating image display device is that when viewing a space-floating image from the user's viewpoint, it may be difficult to sense the sense of distance when recognizing an object image in the depth direction. This may lead to a possibility that the user may misjudge the sense of distance when operating the object image in mid-air, resulting in an erroneous operation. Therefore, in order to prevent an erroneous operation, a partition 3301, which is a physical frame, may be provided. From the user's viewpoint, the partition 3301 allows the user to more clearly sense the sense of distance in the depth direction. The relationship between this partition 3301 and the object image can prevent an erroneous operation of the object image (e.g., the enter button 2901).

[0359] [Preventing Accidental Operation of the Confirm Button (2)] Fig. 34 shows a configuration example of operation unit 1510 in a modified example of partition 3301 of Fig. 33. In front surface 3591 of housing 3590, opening 1515 is provided with frame 3401 (corresponding to frame 1530 in Fig. 20B ), which is a physical frame, surrounding the entire display range 3R of floating-in-space image 3 on all sides. Furthermore, partition 3402 is provided to separate the spatial area within this frame 3401. This partition 3402 divides the area of ​​wheel image 1520 from the area of ​​the image of confirm button 2901. Frame 3401 and partition 3402 have widths in the front and rear directions in the depth direction relative to the position of front surface 3591. The configuration example of Fig. 34 can also prevent accidental operation, as in Fig. 33 .

[0360] FIG. 35 shows a modified example of the divider 3301 in FIG. 33. In this modified example, the front surface 3591 of the housing 3590 has two separated openings 1515, openings 1515-1 and 1515-2. In one opening 1515-1, an image 1520 of a wheel is displayed as part of the floating image in space 3, and in the other opening 1515-2, located above, an image of a confirm button 2901 is displayed as part of the floating image in space 3. Also, each opening 1515 may be provided with a frame with a width in the depth direction as described above. In this modified example, the housing portion 3403 between the two openings 1515 can be said to function in the same way as the divider 3301 or divider 3402 described above.

[0361] [Prevention of Accidental Operation of Confirmation Button (3)] Fig. 36A shows a configuration example of the operation unit 1510 in a modified example of Example 3. In Fig. 36A , a physical confirmation button 3601 is provided on the front surface 3591 of the housing 3590, outside the opening 1515, for example, at an upper position. After the user 230 selects a product by manipulating the wheel image 1520 in the opening 1515 in mid-air with the finger 231, if the user decides to purchase the product, the user moves the finger 231 outside the opening 1515 and presses the confirmation button 3601 as a contact operation. If the user 230 moves the finger 231 toward the upper confirmation button 3601 while operating the wheel image 1520 by sliding, for example, the finger 231 hits the upper edge of the opening 1515 and stops. This prevents the confirmation button 3601 from being accidentally operated.

[0362] As another modification, as shown in Fig. 36B, a physical cancel button 3602 may be provided on the outside of the opening 1515. In this case, when the user 230 wishes to cancel the product selection, the cancel button 3602 is pressed by a touch operation with the finger 231. In response to this pressing, the product vending machine 3000 cancels the product selection state, in other words, resets it, returning it to, for example, an initial state. The user 230 can also start over from the beginning.

[0363] [Prevention of Accidental Operation of the Confirm Button (4)] FIG. 37A is a side view showing an example of the configuration of the operation unit 1510 in a modified example (first example) of Example 3. This first example is conceptually similar to FIG. 20B and the like, but the image of the wheel 1520 and the image of the Confirm button 2901 are formed as the floating-in-space image 3 at a position protruding a distance d1 toward the front surface 3591 of the housing 3590. The protruding distance of the floating-in-space image 3 can be appropriately designed according to the various configuration examples described above. Furthermore, in this example, a partition 3701, which is a physical frame, is provided, as described above ( FIG. 33 , etc.), to prevent accidental operation of the Confirm button 2901. This partition 3701 is located so as to overlap with the floating-in-space image 3 in the depth direction (y direction), i.e., at a distance d1. This partition 3701 has a width d2 in the y direction.

[0364] 37A , in order to position this partition 3701 at the illustrated position in the space, a housing connection part 3702 that connects this partition 3701 to a front surface 3591 of the housing 3590 is also provided. In this example, the partition 3701 is connected to a position above the opening 1515 in the front surface 3591 of the housing 3590 via the housing connection part 3702. One side of the housing connection part 3702 is fixed to the front surface 3591, and the other side is fixed to the partition 3701. For example, a metal plate and screws can be used for the housing connection part 3702, but the present invention is not limited to this.

[0365] Wheel image 1520 is positioned so as to float below partition 3701. If user 230 operates wheel image 1520 with finger 231 and accidentally moves finger 231 toward select button 2901, finger 231 is blocked by partition 3701 (particularly the bottom end of partition 3701), limiting the range of movement. This prevents the select button 2901 from being accidentally operated.

[0366] 37A also illustrates an example of the placement of a sensor 3710 for detecting an aerial operation. In this example, the sensor 3710, for example, the camera or TOF sensor described above, is placed in consideration of being able to detect the wheel image 1520 with high accuracy. The optical axis of the sensor 3710 is placed so as to face the center position of the wheel image 1520, and the detection range of the sensor 3710 covers the wheel image 1520.

[0367] Furthermore, in this example, a sensor 3720, for example, an infrared sensor, separate from the sensor 3710, is provided on the partition 3701. This sensor 3720 only needs to detect an aerial operation (touch operation) of the decision button 2901, and does not need to detect the amount of displacement, as does the sensor 3710. For example, when infrared rays emitted by the infrared sensor that is the sensor 3720 are blocked by the finger 231, this is detected as a press of the decision button 2901. Note that the aerial operation detection sensor 1351 can also be used as the sensor 3720.

[0368] As in this example, the sensor for detecting the mid-air operation of the wheel image 1520 and the sensor for detecting the operation of the enter button 2901 may be configured as separate sensors and detection mechanisms.

[0369] Regarding the positional relationship in the depth direction (y direction) between the structure of the partition 3701 and the spatial floating image 3 and the structural details, several configuration examples can be given as follows, including Fig. 37A. Any of them is applicable.

[0370] FIG. 37B shows a second example. In the second example, the floating-in-space image 3 is formed closer to the partition 3701 in the depth direction (y direction). In other words, the partition 3701 is positioned further back than the position where the floating-in-space image 3 is formed. Distance d3 is the distance from the front surface 3591 to the floating-in-space image 3. Distance d4 is the distance from the front surface 3591 to the partition 3701. Distance d3 is larger (d4<d3). The image light that forms the image of the enter button 2901, emitted from the optical system of the floating-in-space image display device 1000, passes through the space inside the partition 3701 in the y direction and forms the image of the enter button 2901 at the position shown in the figure. More specifically, as shown in the figure, a portion of the image light A1 is blocked by the partition 3701 (particularly the bottom end of the partition 3701), so the formed floating-in-space image 3 is divided into two parts. In the second example, since the floating-in-space image 3 is formed in front of the partition 3701, for example, if an aerial operation detection sensor 1351 is provided on the housing connection part 3702 as shown in the figure, it is possible for the single aerial operation detection sensor 1351 to detect operations on the floating-in-space image 3 without being obstructed by the partition 3701. In other words, it is possible for the single aerial operation detection sensor 1351 to detect operations on both the enter button 2901 and the wheel image 1520.

[0371] In the second example, from the perspective of the user 230, the object image of the floating in space image 3 appears closer to the partition 3701; in other words, the partition 3701 does not appear to be in front of the floating in space image 3. In other words, from the perspective of the user 230, the enter button 2901 appears to have a greater floating feel. The second example has the advantage that vignetting of the object image by the partition 3701, in other words, no blocking of image light, occurs between the viewpoint of the user 230 and the object image of the enter button 2901, etc. Furthermore, in the second example, the presence of the partition 3701 arranged immediately behind the object image can prevent erroneous operation of the enter button 2901. That is, when user 230 touches an object image with finger 231, finger 231 eventually penetrates through floating-in-space image 3 to the far side and interferes with partition 3701 (particularly the bottom end of partition 3701), preventing finger 231 from moving to the object image in the internal space of partition 3701. Therefore, it is possible to prevent erroneous operation of, for example, enter button 2901 in the internal space of partition 3701.

[0372] FIG. 37C shows a third example. In the third example, the floating-in-space image 3 is formed at a position further back than the position of the partition 3701 in the depth direction (y direction). In other words, the partition 3701 is placed at a position closer to the viewer than the position where the floating-in-space image 3 is formed. Distance d5 is the distance from the front surface 3591 to the floating-in-space image 3. Distance d6 is the distance from the front surface 3591 to the partition 3701. Distance d6 is larger (d5<d6). Furthermore, in the third example, the width d7 in the depth direction of the partition 3701 is smaller than in the first example and the like. In the third example, the enter button 2901 is visually recognized at a position slightly further back than the partition 3701 when viewed from the user 230. In the third example, the partition 3701 appears closer to the viewer, and the partition 3701 can also prevent the enter button 2901 from being accidentally pressed. In the third example, since the floating-in-space image 3 is formed further back than the partition 3701, for example, if an aerial operation detection sensor 1351 is provided on the housing connection part 3702 as shown in the figure, it is possible for a single aerial operation detection sensor 1351 to detect operations on the floating-in-space image 3 without being obstructed by the partition 3701. In other words, a single aerial operation detection sensor 1351 can detect operations on both the enter button 2901 and the wheel image 1520. Furthermore, with this arrangement, the aerial operation detection sensor 1351 can be placed on the back side of the housing connection part 3702 as seen from the user, thereby achieving an excellent design in which the user is not aware of the presence of the aerial operation detection sensor 1351.

[0373] In the third example, the partition 3701 is located closer to the object image of the floating image 3, so when the user 230 operates the wheel image 1520 with his / her finger 231 and accidentally moves his / her finger 231 toward the decision button 2901, his / her finger 231 is blocked by the partition 3701 (particularly the bottom end of the partition 3701). Therefore, the mistaken operation of the decision button 2901 can be prevented more reliably.

[0374] The above-described configuration example regarding the positional relationship in the depth direction can be similarly applied to the various configuration examples (FIGS. 20A to 20D, etc.) described above.

[0375] [Modification: Operation in the Depth Direction] In the above-described third embodiment, the aerial operation and detection for the wheel image 1520 are limited to the touch (contact) of the fingers on the x-z plane of the floating-in-space image 3 and directions within the x-z plane, and no aerial operation or detection in the depth direction (y direction) is used. In a modification, aerial operation and detection in the depth direction (y direction) may also be used. When using this, the floating-in-space image display device 1000 is equipped with a sensor capable of detecting aerial operation in the depth direction (y direction) as a user operation detection mechanism. As an example of such a sensor, an imaging unit 1180 such as a camera arranged to capture the movement of the fingers in the depth direction may be used, or multiple aerial operation detection sensors 1351 arranged in the depth direction may be used.

[0376] 38 shows a side view of an example of a push operation as an aerial operation in the depth direction (y direction) in a modified example, and an example of a sensor capable of detecting the push operation. For example, after touching the wheel image 1520, the finger 231 of the user 230 enters in the y direction, which is the depth side. The distance of entry from the position of the floating-in-space image 3 to the depth side is indicated by distance d38. The state of the finger 231 entering the space within the opening 1515 is maintained for a certain period of time or more. When the floating-in-space image display device 1000 detects such a movement of the finger 231, it detects and determines that it is a push operation in the depth direction.

[0377] 38 , as an example of sensors capable of detecting a pressing operation, a plurality of aerial operation detection sensors 1351 are arranged above the floating-in-space image 3. A plurality of aerial operation detection sensors 1351, for example, three aerial operation detection sensors 1351 arranged in front and behind the position of the floating-in-space image 3 as the center, are arranged facing downward. These sensors can detect the position of the fingers 231 (for example, fingertips) in the depth direction (y direction). Furthermore, without being limited to this, even if a camera is installed at the same position facing downward, it can similarly detect the position of the fingers 231 (for example, fingertips) in the depth direction (y direction).

[0378] Furthermore, the floating-in-space image display device 1000 and the product vending machine 3000 may perform predetermined control in response to the detection of an aerial operation in the depth direction (y direction) such as the example of the pushing operation described above. Examples of this predetermined control include the following.

[0379] First Example: As described above, touch and slide operations (operations in the x and z directions) on the wheel image 1520 are assumed to select the product selection button 1502. When the user 230 decides on the product selection, a dedicated decision button (a decision button on the space-floating image 3 or a hardware decision button) may be operated. In a modified example, a dedicated decision button is not used, and an operation in the depth direction on the wheel image 1520, for example, a pressing operation, is accepted, and this pressing operation is received as a "decision" operation. For example, when the space-floating image display device 1000 detects this pressing operation, it executes a purchase decision process for the selected product. In other words, when a certain product selection button 1502 is selected and this "decision" operation is performed, the process transitions to purchasing the product corresponding to that selected product selection button 1502. In this modified example, the user 230 can make the purchase decision by simply manipulating the wheel image 1520 in the air, without having to operate a separate dedicated decision button or the like.

[0380] Second Example: Touch and slide operations (operations in the x and z directions) on the wheel image 1520 are assumed to select the product selection button 1502, as described above. When the user 230 wants to cancel (i.e., reset) the product selection, a dedicated button (a cancel button on the floating-in-space image 3 or a hardware cancel button) may be operated. In a modified example, instead of using a dedicated cancel button, an operation in the depth direction on the wheel image 1520, such as a pressing operation, is accepted, and this pressing operation is treated as a "cancel" or "reset" operation. For example, when the floating-in-space image display device 1000 detects this pressing operation, it executes a process to cancel the product selection and returns to the initial state regarding the product selection. The initial state is a state in which no product selection button 1502 is selected, or a state in which a product selection button 1502 in a predetermined default position is selected. In this modified example, the user 230 can cancel the product selection by simply operating the wheel image 1520 in mid-air, without operating a separate dedicated cancel button or the like.

[0381] [Modification: Product Image Display] The following modification of Example 3 is also possible. FIG. 39 shows the operation unit 1510 in this modification. In this modification, the floating-in-space image 3 of the operation unit 1510 displays, together with the wheel image 1520, a product image 3901 associated with the product selection button 1502 that is selected in response to an aerial manipulation of the image 1520. In this example, in the display range 3R of the floating-in-space image 3, the wheel image 1520 is displayed at the bottom, and the product image 3901 is displayed at the top. The effect of an aerial manipulation on the wheel image 1520 is the same as described above. In response to an aerial manipulation, for example, the product selection button 1502X for a certain product X (e.g., a beverage) is selected, and the product selection button 1502X lights up. At the same time, the product image 3901 associated with the product selection button 1502X is displayed. The image 3901 may include information such as a product image, product number, product name, product description, price, and advertisement. As user 230 continues to manipulate wheel image 1520 in mid-air, the display content of product image 3901 changes in accordance with the resulting change in selection state. User 230 can decide on the desired product to purchase by looking at this image 3901 as well as product display section 1501. After the product to purchase has been decided, the state in which product image 3901 is displayed corresponds to the state in which product selection button 1502 corresponding to that product is selected. Therefore, user 230 then simply operates decision button 2901 and proceeds with the payment procedure, as described above.

[0382] In this modified example, it can be said that the product image 3901, which is part of the floating image 3, also functions as the product display unit 1501. In the case of this modified example, when the user 230 performs an aerial operation on the wheel image 1520, it is possible to omit looking at the product on the product display unit 1501 to check it.

[0383] In the example of Figure 39, an image showing the product number of the product selected by manipulating the wheel image 1520 in the air may be displayed in a part of the floating image 3, or on a display unit separate from the floating image 3.

[0384] <Embodiment 4> As embodiment 4 of the present invention, a configuration example of a space floating image display device will be described. The basic configuration of embodiment 4 is the same as embodiment 3. In embodiment 4, differences from embodiment 3 will be described, and repeated explanations of the same configuration as embodiment 3 will be omitted.

[0385] The space-floating image display device of Example 4 is applied to an automatic ticket vending machine installed in a public facility such as a station or a store, and is provided as one of the components in the automatic ticket vending machine. This space-floating image display device is provided as a UI for operation and input such as selecting items / buttons from a menu screen for purchasing tickets, etc. In other words, this space-floating image display device functions as the operation unit, input unit, display unit, and UI unit of the automatic ticket vending machine.

[0386] In Example 4 shown in FIG. 40 etc., similar to Example 3, a wheel image 4020 is displayed on the operation unit 4010 of the automatic ticket vending machine 4000, and any object image (e.g., a menu item or a button) displayed on the screen of the display 4001 can be selected and confirmed in response to a mid-air operation by the user 230 on the wheel image 4020. Also, in Example 4, similar to Example 3, a confirmation button or the like may be displayed in the floating-in-space image 3 of the operation unit 4010. Also, in Example 4, similar to Example 3, an object image (e.g., a confirmation button) for which erroneous operation should be prevented may be configured to be surrounded by a partition that is a physical frame. Also, in Example 4, similar to Example 3, an opening may be separated between the wheel image 4020 and the confirmation button image.

[0387] The automatic ticket vending machine 4000 may be a device such as a fare adjustment machine, a reception machine, etc. The present invention is not limited to the automatic ticket vending machine 4000, and can be similarly applied to a predetermined device that requires an item / button selection operation on a display screen.

[0388] [Ticket Vending Machine] Fig. 40 shows an example of the configuration of an automatic ticket vending machine 4000 equipped with the space-floating image display device 1000 of Example 4 as an operation unit 4010. The operation unit 4010 may be equipped with any of the space-floating image display devices 1000 described in Example 1 or Example 2. Hereinafter, in various descriptions of the automatic ticket vending machine of this embodiment, the space-floating image display device 1000 which is the operation unit 4010 has, for example, the configuration shown in Fig. 3. The operation of each component of the space-floating image display device 1000 which is the operation unit 4010 is controlled by the control unit 1110 as described in Fig. 3. The image display operation of the space-floating image display device 1000 which is the operation unit 4010 is displayed by the display device 1 based on the control of the image control unit 1160 as described in Fig. 3. As described in FIG. 3 , the operation detection operation of the air-floating image display device 1000, which is the operation unit 4010, is performed by the air-operation detection unit 1350 based on the sensing results of the air-operation detection sensor 1351, and generates control information based on the operation detection. The functions and operations of the other components shown in FIG. 3 are the same as those described in Example 1 or Example 2, so repeated explanations will be omitted. FIG. 40 shows a perspective view of the front surface 4091 of the housing 4090 of the automatic ticket vending machine 4000 as seen from the user 230. The display 4001, the operation unit 4010, etc. are provided on the front surface 4091 of the upper part of the housing 4090 of the automatic ticket vending machine 4000. In addition, the front surface of the lower part of the housing 4090 is provided with a ticket dispenser slot 4092, a payment device (not shown), etc.

[0389] 40 has a vertically standing housing 4090, and is provided with a display 4001 (e.g., a liquid crystal display panel) having a two-dimensional screen on an upper front surface 4091. The upper front surface 4091 and the screen of the display 4001 are slightly tilted so that they are easily visible to the user 230.

[0390] Furthermore, on the front surface 4091, an operation unit 4010 by the space floating image display device 1000 is provided at a position below the display 4001. In the operation unit 4010, a space floating image 3 is formed in an opening 4015, and a wheel image 4020 by the space floating image 3 is displayed. The wheel image 4020 is the same as the image 1520 in Example 3. The user 230 performs an operation in the air on the wheel image 4020 with his / her fingers 231.

[0391] [Configuration Example of Ticket Vending Machine] The configuration example of FIG. 16 can be similarly applied to the automatic ticket vending machine 4000 of FIG. 40. The product vending machine main body 3500 in FIG. 16 is replaced with the automatic ticket vending machine main body, and the product vending machine function unit 3510 is replaced with the automatic ticket vending machine function unit. Note that, even in the example of the automatic ticket vending machine 4000, the space-floating image display device 1000, which is the operation unit 4010, communicates with the communication interface of the automatic ticket vending machine main body via the communication unit 1132 under the control of the control unit 1110 of FIG. 3. Through this communication, various control information, such as control information based on the detection of aerial operations, can be transmitted from the space-floating image display device 1000 to the automatic ticket vending machine main body. Note that the external power input interface 1111 of the space-floating image display device 1000, which is the operation unit 4010, shown in FIG. 3, is connected to the power supply of the automatic ticket vending machine main body. This ensures power to operate the space-floating image display device 1000, which is the operation unit 4010.

[0392] Fig. 41 shows an example of the configuration of the ticket vending machine function unit 4510 of the ticket vending machine 4000. Components such as the display 4001 in Fig. 41 are connected to the control unit 4501 of the ticket vending machine 4000. The ticket vending machine function unit 4510 is a component for realizing the functions of the ticket vending machine 4000. Publicly known technology can be applied to the ticket vending machine function unit 4510. The ticket vending machine function unit 4510 includes a display 4001, a video processing unit 4101, an electronic information medium reader / writer 4102, an imaging unit 4103, an audio output unit 4104, an audio input unit 4105, a coin / banknote processing unit 4106, a ticket issuing unit 4107, etc., which are interconnected by an architecture such as a bus.

[0393] The display 4001 is configured, for example, with a liquid crystal display panel (which may in particular be a touch panel), and displays images / videos on the screen of the display, such as a menu screen for ticket purchase guidance, etc. The menu screen displays items / buttons that can be selected by mid-air operation (see FIG. 42, described below).

[0394] The video processing unit 4101 processes video information / video data, creates data / signals for displaying video on the screen of the display 4001, and displays video on the screen of the display 4001 based on the data / signals.

[0395] The electronic information medium reader / writer 4102 is a device that reads and writes data from and to an electronic information medium, and functions as a payment device 4102. The payment device 4102 reads and writes data, for example, from an IC card of the user 230 or information via NFC on a smartphone. In this embodiment, the payment device 4102 using the electronic information medium reader / writer 4102 is provided as part of the automatic ticket vending machine function unit 4510. In this embodiment, the control unit 4501 mainly performs the cooperation and control of the operation unit 4010 with the space-floating image display device 1000. Without being limited to this, for example, the payment device 4102 may cooperate and control the space-floating image display device 1000.

[0396] The imaging unit 4103 uses a camera to capture an image of the vicinity of the housing 4090, for example, the vicinity of the front surface. As a result, the imaging unit 4103 also functions as a human presence sensor that detects the user 230 standing in front. The imaging unit 4103 may be realized as part of the payment device 4102. User authentication, etc. may be performed using the imaging unit 4103.

[0397] The audio output unit 4104 outputs a predetermined sound from a speaker. The audio input unit 4105 inputs the voice of the user 230 through a microphone. The coin / bill processing unit 4106 processes coins or bills inserted by the user 230 into the coin / bill insertion unit. The ticket issuing unit 4107 issues purchased tickets and outputs them to the ticket outlet 4092.

[0398] The communication interface 4505 is a communication interface of the ticket vending machine main body, and corresponds to the communication interface 3505 in the configuration example of the product vending machine main body 3500 in FIG. 16 . The communication interface 4505 communicates with the space-floating image display device 1000, which is the operation unit 4010. Control information based on the detection of an aerial operation from the space-floating image display device 1000, which is the operation unit 4010 (hereinafter referred to as detection information / control information), is transmitted from the space-floating image display device 1000, which is the operation unit 4010, to the control unit 4501 via this communication. The control unit 4501 will be described later. In various explanations of the ticket vending machine of this embodiment, the user's aerial operation in the control unit 4501 is reflected in the operation or processing of the ticket vending machine via this communication and the control of the control unit 4501.

[0399] The control unit 4501 is a control unit of the ticket vending machine main body, and corresponds to the control unit 3501 in the example configuration of the product vending machine main body 3500 in FIG. 16 . The control unit 4501 controls the items / buttons on the display 4001 based on cooperation via communication between the operation unit 4010 and the space-floating image display device 1000. For example, the control unit 4501 transmits a control signal 4111 to the image processing unit 4101, and the image processing unit 4101 performs image processing in accordance with the signal 4111. This controls the display of the items / buttons on the display 4001. The signal 4111 is, for example, a signal for highlighting an item / button that is selected by an operation in the air in the image 4020, and is a signal that instructs the on / off of the highlighting, etc. The selected item / button is highlighted based on the signal 4111.

[0400] [Ticket Vending Machine Operation Unit: Aerial Operation / Display Control Example (1)] FIG. 42 shows an example of aerial operation and display control for a wheel image 4020 on the operation unit 4010 of the ticket vending machine 4000. In state A, a GUI (Graphical User Interface) image, such as menu items for purchasing tickets, is displayed on the screen of the display 4001. The content of this screen is arbitrary, and publicly known techniques can be similarly applied. This example is an example of a screen for purchasing concert tickets. This screen displays a guide such as "Please select the ticket you wish to purchase," and has an item 4201 for selecting and purchasing from each ticket (e.g., A to D), in other words, a ticket purchase button 4201. In addition, a confirmation button and a cancel button for purchase may also be provided. Audio output may also be provided along with the display.

[0401] State A is the initial state before any selection has been made by the user 230. In the initial state, no item / button is selected. Alternatively, a predetermined item / button, for example, the ticket purchase button 4201 for "Ticket A" at the top, may be selected as the default (default position) item / button selection in the initial state.

[0402] When the display 4001 is a touch panel, the user 230 can also perform a selection operation by directly touching an item / button on the screen, as in the past, and such a touch operation is also permitted. A selection operation by touching the item / button and a selection operation by manipulating the image 4020 in the air coexist.

[0403] State B corresponds to the screen of state A, and is a state in which a wheel image 4020 and the like are displayed on the floating-in-space image 3 in the opening 4015 of the operation unit 4010. The user 230 operates the wheel image 4020 by touching or sliding the wheel image 4020 with the finger 231. In the floating-in-space image 3, not only the wheel image 4020 but also a guide image such as "Turn to select" may be displayed, and the guide content may be simultaneously output as audio. In contrast to the selection of a product (corresponding product selection button 1502) in Example 3, in Example 4, a ticket (corresponding menu item, ticket purchase button 4201) is selected.

[0404] In state B, assume that the user 230 performs a sliding operation, for example, from top to bottom (-z direction), as an aerial operation on the wheel image 4020. The space-floating image display device 1000 detects the aerial operation and obtains detection information such as the amount of displacement and the direction of displacement. The space-floating image display device 1000 performs mapping (association) with items / buttons on the screen of the display 4001 based on the detection information of the aerial operation, similar to, for example, Example 3. The space-floating image display device 1000 selects an item / button according to the amount of displacement, etc., and transmits control information representing the selected item / button to the control unit 4501 of the ticket vending machine 4000.

[0405] In Fig. 43, state C shows a change in the display screen of the display 4001 corresponding to the mid-air operation of state B. The control unit 4501 of the automatic ticket vending machine 4000, in cooperation with the space-floating image display device 1000, controls the display 4001 and the image processing unit 4101 so as to highlight the selected item / button. In this example, in response to the slide operation in the z direction, the selected ticket purchase button 4201 changes from the "Ticket A" purchase button to the "Ticket D" purchase button. In state C, the "Ticket D" purchase button is in a selected state and is highlighted with a colored frame or the like. This highlighting indicates the selected state of the item / button. The display mode when the item / button 4201 is selected is not limited to this highlighting, and other modes may be used.

[0406] In this way, in the fourth embodiment, as in the third embodiment, the selection state of the object image (e.g., item / button 4201) on the screen of the display 4001 is controlled to change in accordance with the detection result (e.g., displacement amount) of the mid-air operation of the wheel image 4020. The control method is the same as in the third embodiment.

[0407] In this example, since the multiple ticket purchase buttons 4201 are arranged vertically (z direction) as multiple object images on the screen of display 4001, aerial manipulation in the vertical direction (z direction) is effective. Similarly, if the objects on the screen are arranged horizontally (x direction), aerial manipulation in the horizontal direction (x direction) is effective. Although not shown, when selecting from the enter button and cancel button arranged horizontally (x direction) on the screen of display 4001, aerial manipulation in the horizontal direction (x direction) can be added.

[0408] In this example (state A), multiple ticket purchase buttons 4201 are arranged vertically (z direction) as multiple object images on the screen of display 4001, so correspondingly, the aerial operations accepted for wheel image 4020 may be limited to only aerial operations in the vertical direction (z direction). For example, the space floating image display device 1000 may detect the amount of displacement in the z direction when detecting an aerial operation for image 4020. Furthermore, when limiting acceptance of aerial operations to only those in a specific direction, the display mode of wheel image 4020 may be changed to express this restriction. For example, by displaying an image like state D in FIG. 21 or state H in FIG. 22, the user 230 may be informed that only aerial operations in a specific direction are accepted.

[0409] In addition, in the example of state C in FIG. 43 , a path in the z direction is configured from the "Ticket A" purchase button to the "Ticket D" purchase button in relation to the change in the selection state. In the fourth embodiment, as in the third embodiment, it is possible to control the highlighting of this path. For example, on the path from the "Ticket A" purchase button to the "Ticket D" purchase button, it is also possible to control the highlighting of the "Ticket B" purchase button and the "Ticket C" purchase button, which are passed through along the way. This allows the user 230 to understand the change in the selection state more clearly.

[0410] FIG. 44 shows another example of the screen configuration of the display 4001. Multiple items 4401 are arranged vertically and horizontally on the screen as multiple object images. In this example, items A through L are arranged in four rows and three columns. In this case, a path can be considered on the x-z plane from an item / button in a first selection state to an item / button in a second selection state. Controls such as sequentially highlighting items / buttons along the path can also be applied. For example, the diagram shows a case in which a path from item A in a first selection state to item L in a second selection state is configured by performing a downward operation and a rightward operation on the image 4020. In this case, control is possible such that items (D, G, J, K) along the path from item A in the first selection state to item L in the second selection state are sequentially highlighted, and finally item L in the second selection state is highlighted.

[0411] Next, state D in Fig. 43 is a continuation from state C. In state D, for example, when the "Ticket D" purchase button is selected as in state C, the space-floating image display device 1000 switches and updates the display content of the space-floating image 3 according to the selection state. State D shows an example of the display after switching. In the space-floating image 3, a decision button 4301 is displayed as part of the image. In this example, the decision button 4301 is displayed above the wheel image 4020, but this is not limiting. Also, instead of just the decision button 4301, it is also possible to output a guide such as "When you have decided which ticket to purchase, press the decision button."

[0412] In the case where the user...

Claims

1. An airborne floating video display device mounted on a vending machine, the airborne floating video display device comprising: a video processing unit that performs video processing; a display unit that displays the video processed by the video processing unit; an optical system that generates an airborne floating video based on the video displayed by the display unit; a user operation detection mechanism that detects an operation by a user with respect to the display range of the airborne floating video; a control unit; and a communication interface. The airborne floating video display device constitutes an operation unit for operating the vending machine, the airborne floating video display device displays the airborne floating video for operating the vending machine, and the control unit of the airborne floating video display device outputs, via communication of the communication interface, information for causing the vending machine to execute a predetermined process based on detection of an operation on the airborne floating video. The predetermined process includes a process of selecting one from a plurality of candidates, and the airborne floating video includes an object video that receives an operation for the selection. An airborne floating video display device.

2. The airborne floating video display device according to claim 1, wherein the vending machine is a commodity vending machine or a ticket vending machine. An airborne floating video display device.

3. The airborne floating video display device according to claim 1, wherein the airborne floating video display device displays a video that receives an operation in a direction within a plane of the airborne floating video in the airborne floating video. An airborne floating video display device.

4. The airborne floating video display device according to claim 3, wherein the video is a video of a wheel or a sphere. An airborne floating video display device.

5. The airborne floating video display device according to claim 1, wherein the control unit of the airborne floating video display device performs control to output, via communication of the communication interface, information for determining a selection in the predetermined process based on detection information of the operation on the airborne floating video. An airborne floating video display device.

6. The airborne floating video display device according to claim 2, wherein the predetermined process includes a process related to selection and purchase of a commodity of the commodity vending machine or a process related to selection and ticket issuance of the ticket vending machine. An airborne floating video display device.

7. In the airborne video display device according to claim 2, the predetermined process includes processes related to the selection and purchase of products in the vending machine, and the predetermined process includes a process of selecting one product selection button from a plurality of product selection buttons on the product bulletin board or display of the vending machine. An airborne video display device.

8. In the airborne video display device according to claim 7, the control unit of the airborne video display device controls to output information for lighting the selected product selection button to the vending machine via the communication of the communication interface. An airborne video display device.

9. In the airborne video display device according to claim 8, the information for lighting the selected product selection button is information for controlling to light the product selection button in a path from a first product selection button to a second product selection button selected when lighting the selected product selection button in the vending machine. An airborne video display device.

10. In the airborne video display device according to claim 1, the operation unit included in the airborne video display device forms the airborne video in an opening on the front surface of the housing of the vending machine, and the airborne video is formed at a position corresponding to the position of the front surface in the depth direction or at a position extending forward from the position of the front surface. An airborne video display device.

11. In the airborne video display device according to claim 1, at a first time point, the airborne video display device displays a first video that accepts an operation in a direction within the plane of the airborne video in the airborne video, prompts the user to perform an operation on the first video, and causes the user to make a first selection decision in the predetermined process. After the first selection decision, at a second time point, the airborne video display device displays a second video for selection decision in the airborne video, prompts the user to perform an operation on the second video, and causes the user to make a second selection decision. An airborne video display device.

12. In the airborne video display device according to claim 1, the operation unit includes a partition for physically dividing a plurality of object videos displayed in the airborne video on the front surface of the housing of the vending machine. An airborne video display device.

13. In the airborne floating image display device according to claim 1, the operation unit includes a plurality of openings on the front surface of the housing of the vending machine for physically dividing a plurality of object images displayed in the airborne floating image. An airborne floating image display device.

14. In the airborne floating image display device according to claim 12, the partition is connected via a housing connection portion at a position extending forward from the front surface of the housing of the vending machine. An airborne floating image display device.

15. In the airborne floating image display device according to claim 12, the partition is arranged at a position in the depth direction corresponding to the formation position of the airborne floating image with respect to the front surface of the housing of the vending machine. An airborne floating image display device.

16. In the airborne floating image display device according to claim 12, the partition is arranged at a position deeper than the formation position of the airborne floating image in the depth direction with respect to the front surface of the housing of the vending machine. An airborne floating image display device.

17. In the airborne floating image display device according to claim 12, the partition is arranged at a position closer to the front side than the formation position of the airborne floating image in the depth direction with respect to the front surface of the housing of the vending machine. An airborne floating image display device.

18. In the airborne floating image display device according to claim 2, the predetermined process includes a process related to the selection and issuance of tickets of the ticket vending machine. The predetermined process includes a process of selecting one item or button from a plurality of items or buttons on the display of the ticket vending machine. An airborne floating image display device.

19. In the airborne floating image display device according to claim 18, the operation unit includes a partition on the front surface of the housing of the ticket vending machine for physically dividing a plurality of object images displayed in the airborne floating image. An airborne floating image display device.

20. In the airborne video display device according to claim 2, the predetermined process includes processes related to ticket selection and ticket issuance of the vending machine, the operation unit displays, in the airborne video, a screen having a plurality of items or buttons for operations related to ticket selection and ticket issuance of the vending machine, and based on detection of an operation on the screen of the airborne video, as the predetermined process, executes a process of selecting one item or button from the plurality of items or buttons. An airborne video display device.

21. In the airborne video display device according to claim 19, the operation unit includes a partition for physically dividing a plurality of object videos displayed in the airborne video on the front surface of the housing of the vending machine. An airborne video display device.

22. A merchandise vending machine, comprising an airborne video display device mounted on the housing of the merchandise vending machine, the airborne video display device including a video processing unit that performs video processing, a display unit that displays the video processed by the video processing unit, an optical system that generates an airborne video based on the video displayed by the display unit, and a user operation detection mechanism that detects an operation by a user on the display range of the airborne video, the airborne video display device constitutes an operation unit for operating the merchandise vending machine, the operation unit displays the airborne video for operating the merchandise vending machine, the operation unit causes the merchandise vending machine to execute a predetermined process based on detection of an operation on the airborne video, the predetermined process includes a process of selecting one from a plurality of candidates, and the airborne video includes an object video that accepts an operation for the selection. A merchandise vending machine.

23. A display device comprising an airborne floating image display device mounted on a housing of the display device, the airborne floating image display device including an image processing unit that performs image processing, a display unit that displays an image processed by the image processing unit, an optical system that generates an airborne floating image based on the image displayed by the display unit, and a user operation detection mechanism that detects an operation by a user with respect to a display range of the airborne floating image, the airborne floating image display device constituting an operation unit for operating the display device, the operation unit displaying the airborne floating image for operating the display device, the operation unit causing the display device to execute a predetermined process based on detection of an operation with respect to the airborne floating image, the predetermined process including a process of selecting one from a plurality of candidates, and the airborne floating image including an object image that receives an operation for the selection, the display device.

Images