Aerial video display device, product vending machine, and display device

Abstract

To provide a more suitable aerial video display device, which contributes to the Sustainable Development Goals (SDGs) of "3 Good health and well-being", "9 Industry, innovation and infrastructure" and "11 Sustainable cities and communities".SOLUTION: An aerial video display device mounted on a product vending machine or a ticket vending machine constitutes an operation unit for operating the product vending machine or the ticket vending machine. The operation unit displays an aerial image for operating the product vending machine or the ticket vending machine, and, on the basis of detection of an operation on the aerial image, causes the product vending machine or the ticket vending machine to execute a predetermined process. The predetermined process includes a process for selecting one of a plurality of candidates. The aerial image includes an object image for receiving an operation for selection.SELECTED DRAWING: Figure 15

Description

Technical Field

[0001] The present invention relates to an airborne floating image display device.

Background Art

[0002] Regarding airborne floating information display technology, for example, it is disclosed in Patent Document 1.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the disclosure of Patent Document 1, the consideration regarding the configuration for obtaining practical brightness and quality of the airborne floating image, and the configuration for the user to visually recognize the airborne floating image more enjoyably is not sufficient.

[0005] An object of the present invention is to provide a more suitable airborne floating image display device.

Means for Solving the Problems

[0006] To solve the above problems, for example, the configuration described in the claims is adopted. This application includes a plurality of means for solving the above problems. If an example is given, it may be configured as follows. An airborne floating video display device mounted on a vending machine or a ticket vending machine, wherein the airborne floating video display device includes a video processing unit that performs video processing, a display unit that displays the video processed by the video processing unit, an optical system that generates an airborne floating video based on the video displayed by the display unit, and a user operation detection mechanism that detects an operation by a user with respect to the display range of the airborne floating video. The airborne floating video display device constitutes an operation unit for operating the vending machine or the ticket vending machine. The operation unit displays the airborne floating video for operating the vending machine or the ticket vending machine. Based on detection of an operation on the airborne floating video, the operation unit causes the vending machine or the ticket vending machine to execute a predetermined process. The predetermined process includes a process of selecting one from a plurality of candidates. The airborne floating video includes an object video that accepts an operation for the selection.

Effect of the Invention

[0007] According to the present invention, a more suitable airborne floating video display device can be realized. Other problems, configurations, and effects will be clarified in the following description of the embodiments.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 2D

Figure 2E

Figure 2F

Figure 2G

Figure 2H

Figure 2I

Figure 3

Figure 4A

Figure 4B

Figure 4C

Figure 4D

Figure 4E

Figure 4F

Figure 4G

Figure 4H

Figure 4I

Figure 4J

Figure 4K

Figure 4L

Figure 4M

Figure 4N

Figure 4O

Figure 4P

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13A

Figure 13B

Figure 13C

Figure 13D

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20A

Figure 20B

Figure 20C

Figure 20D

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31A

Figure 31B

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36A

Figure 36B

Figure 37A

Figure 37B

Figure 37C

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

Figure 46

Figure 47A

Figure 47B

Figure 47C

Figure 48

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. It should be noted that the present invention is not limited to the description of the embodiments, and various changes and modifications can be made by those skilled in the art within the scope of the technical idea disclosed in this specification. Also, in all the drawings for explaining the present invention, those having the same function are denoted by the same reference numerals, and the repeated description thereof may be omitted.

[0010] The following embodiments relate to a video display device capable of transmitting a video by video light from a video light source through a transparent member that partitions a space such as glass and displaying it as a space floating video outside the transparent member. In the description of the following embodiments, a video floating in space is expressed by the term "space floating video". Instead of this term, it may be expressed as "aerial image", "space image", "aerial floating video", "spatially floating optical image of the display video", "aerial floating optical image of the display video", etc. The term "space floating video" mainly used in the description of the embodiments is used as a representative example of these terms.

[0011] According to the following embodiments, for example, a video display device suitable for use in bank ATMs, station ticket vending machines, digital signage, etc. can be realized. For example, currently, in bank ATMs, station ticket vending machines, etc., a touch panel is usually used. However, by using a transparent glass surface or a light-transmissive plate material, high-resolution video information can be displayed in a state of floating in space on this glass surface or light-transmissive plate material. At this time, by reducing the divergence angle of the emitted video light, that is, making it an acute angle, and further aligning it with a specific polarization, only the regular reflected light can be efficiently reflected to the retroreflective plate. Therefore, the light utilization efficiency is high, and it is possible to suppress ghost images that occur in addition to the main space floating image, which has been a problem in the conventional retroreflective method, and obtain a clear space floating video. Also, an apparatus including the light source of this embodiment can provide a novel and highly usable space floating video display device (space floating video display system) capable of significantly reducing power consumption. Further, for example, it is possible to provide a vehicle space floating video display device capable of one-way space floating video display that is visible inside and / or outside the vehicle.

[0012] <Example 1> Hereinafter, as Example 1 of the present invention, a configuration example of a spatial floating image display device will be described.

[0013] <An example of the usage form of the spatial floating image display device> FIG. 1 is a diagram showing an example of the usage form of a spatial floating image display device according to an embodiment of the present invention, and is a diagram showing the overall configuration of the spatial floating image display device according to this embodiment. Regarding the specific configuration of the spatial floating image display device, it will be described in detail using FIG. 2 and the like. However, light with a specific polarization and a diverging angular characteristic is emitted as an image light beam from the image display device 1, once enters the retroreflective plate 2 through reflection and the like in the optical system within the spatial floating image display device, is retroreflected, passes through a transparent member 100 (such as glass), and forms a real image, an aerial image (spatial floating image 3), outside the glass surface. In the following examples, the retroreflective plate 2 (retroreflective sheet) will be used as an example of the retroreflective member. However, the retroreflective plate 2 of the present invention is not limited to a planar plate, and is used as an example of a concept including a sheet-like retroreflective member attached to a planar or non-planar member, or the entire assembly with a sheet-like retroreflective member attached to a planar or non-planar member. Further, since the light rays after reflection by the retroreflective plate 2 have an imaging optical characteristic, the retroreflective plate 2 may also be expressed as an imaging optical member or an imaging optical plate.

[0014] Also, in a store or the like, a space is partitioned by a show window (also referred to as "window glass") 105, which is a translucent member such as glass. According to the spatial floating image display device of this embodiment, it is possible to display a floating image in one direction with respect to the outside and / or inside of the store (space) through such a transparent member.

[0015] In FIG. 1, the inside (inside the store) of the window glass 105 is shown in the depth direction, and the outside (for example, the sidewalk) is in the foreground. On the other hand, by providing means for reflecting a specific polarization on the window glass 105, it is also possible to form an aerial image at a desired position inside the store.

[0016] <Configuration Example of the Optical System of a Spatial Floating Image Display Device> FIG. 2A is a diagram showing an example of the configuration of the optical system of a spatial floating image display device according to an embodiment of the present invention. Using FIG. 2A, the configuration of the spatial floating image display device will be described more specifically. As shown in FIG. 2A(1), in an oblique direction of a transparent member 100 such as glass, a display device 1 that diverges the image light of a specific polarization at an included angle is provided. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates light of a specific polarization having an included angle diffusion characteristic.

[0017] The image light of a specific polarization from the display device 1 is reflected by a polarization separation member 101 having a film that selectively reflects the image light of a specific polarization provided on the transparent member 100 (in the figure, the polarization separation member 101 is formed in a sheet shape and adhered to the transparent member 100), and is incident on the retroreflector 2. A λ / 4 plate 21 is provided on the image light incident surface of the retroreflector 2. The image light is polarization-converted from a specific polarization to the other polarization by passing through the λ / 4 plate 21 twice, when entering and exiting the retroreflector 2. Here, since the polarization separation member 101 that selectively reflects the image light of a specific polarization has the property of transmitting the polarization of the other polarization that has been polarization-converted, the image light of the specific polarization after polarization conversion passes through the polarization separation member 101. The image light that has passed through the polarization separation member 101 forms a spatial floating image 3 that is a real image outside the transparent member 100. Note that FIG. 2A depicts an example in which the principal ray of the image light incident on the retroreflector 2 is incident at 90° with respect to the retroreflector 2. However, the incident angle of the principal ray of the image light with respect to the retroreflector 2 is not limited to 90°, and for example, an angle of 90° ± 15° can also be used.

[0018] Here, a first example of the polarization design in the optical system of FIG. 2A will be described. For example, the configuration may be such that the video light of S-polarization (S is an abbreviation of senkrecht. Polarization in which the electric field vibrates perpendicular to the incident plane) is emitted from the display device 1 to the polarization separation member 101, and the polarization separation member 101 has the characteristic of reflecting S-polarization and transmitting P-polarization (P is an abbreviation of parallel. Polarization in which the electric field vibrates in the incident plane). In this case, the S-polarized video light that has reached the polarization separation member 101 from the display device 1 is reflected by the polarization separation member 101 and travels toward the retroreflector 2. When the video light is reflected by the retroreflector 2, since it passes through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2 twice, the video light is converted from S-polarization to P-polarization. The video light converted to P-polarization travels toward the polarization separation member 101 again. Here, since the polarization separation member 101 has the characteristic of reflecting S-polarization and transmitting P-polarization, the P-polarized video light passes through the polarization separation member 101 and then through the transparent member 100. Since the video light that has passed through the transparent member 100 is the light generated by the retroreflector 2, a spatial floating image 3, which is an optical image of the display image of the display device 1, is formed at a position that has a mirror relationship with the display image of the display device 1 with respect to the polarization separation member 101. Such a polarization design can preferably form the spatial floating image 3.

[0019] Next, a second example of the polarization design in the optical system of FIG. 2A will be described. For example, the configuration may be such that P-polarized video light is emitted from the display device 1 to the polarization separation member 101, and the polarization separation member 101 has the property of reflecting P-polarization and transmitting S-polarization. In this case, the P-polarized video light that has reached the polarization separation member 101 from the display device 1 is reflected by the polarization separation member 101 and travels toward the retroreflector 2. When the video light is reflected by the retroreflector 2, since it passes through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2 twice, the video light is converted from P-polarization to S-polarization. The video light converted to S-polarization travels toward the polarization separation member 101 again. Here, since the polarization separation member 101 has the property of reflecting P-polarization and transmitting S-polarization, the S-polarized video light passes through the polarization separation member 101 and then through the transparent member 100. Since the video light that has passed through the transparent member 100 is the light generated by the retroreflector 2, a spatial floating image 3, which is an optical image of the display image of the display device 1, is formed at a position that has a mirror image relationship with the display image of the display device 1 with respect to the polarization separation member 101. Such a polarization design can preferably form the spatial floating image 3.

[0020] Note that the light forming the spatial floating image 3 is a collection of light rays converging from the retroreflector 2 to the optical image of the spatial floating image 3, and these light rays continue to travel straight even after passing through the optical image of the spatial floating image 3. Therefore, unlike the diffused video light formed on a screen by a general projector or the like, the spatial floating image 3 is an image with high directivity. Thus, in the configuration of FIG. 2A, when viewed by the user from the direction of arrow A, the spatial floating image 3 is viewed as a bright image. However, when viewed by another person from the direction of arrow B, the spatial floating image 3 cannot be viewed as an image at all. This characteristic is very suitable for use in a system that displays an image that requires high security or an image with high confidentiality that needs to be hidden from a person facing the user.

[0021] Depending on the performance of the retroreflector 2, the polarization axes of the video light after reflection may become uneven. Also, the reflection angles may become uneven. Such uneven light may not maintain the polarization state and propagation angle assumed in the design. For example, light with a polarization state and propagation angle outside the design assumptions may directly re-enter the video display surface side of the liquid crystal display panel 11 from the position of the retroreflector 2 without passing through the polarization separation member. Light with a polarization state and propagation angle outside the design assumptions may also re-enter the video display surface side of the liquid crystal display panel 11 after being reflected by components within the spatial floating video display device. The light that re-enters the video display surface side of such a liquid crystal display panel 11 may be re-reflected by the video display surface of the liquid crystal display panel 11 that constitutes the display device 1, potentially generating a ghost image and degrading the image quality of the spatial floating image. Therefore, in this embodiment, an absorption-type polarizing plate 12 may be provided on the video display surface of the display device 1. The video light emitted from the display device 1 is transmitted through the absorption-type polarizing plate 12, and the reflected light returning from the polarization separation member 101 is absorbed by the absorption-type polarizing plate 12, thereby suppressing the above-mentioned re-reflection. As a result, it is possible to prevent degradation of the image quality due to ghost images of the spatial floating image. Specifically, if the configuration is such that S-polarized video light is emitted from the display device 1 to the polarization separation member 101, the absorption-type polarizing plate 12 may be a polarizing plate that absorbs P-polarization. Also, if the configuration is such that P-polarized video light is emitted from the display device 1 to the polarization separation member 101, the absorption-type polarizing plate 12 may be a polarizing plate that absorbs S-polarization.

[0022] The above-described polarization separation member 101 may be formed of, for example, a reflective polarizing plate or a metal multilayer film that reflects a specific polarization wave.

[0023] Next, FIG. 2A(2) shows an example of the surface shape of a representative retroreflector 2 as the retroreflector 2. A prism body in which reflecting surfaces of concave portions in the shape of regular triangular pyramids are arranged is disposed on the retroreflector 2. Light rays incident on the arranged concave portions in the shape of triangular pyramids are reflected by a plurality of reflecting surfaces of the concave portions in the shape of triangular pyramids and exit as retroreflected light in a direction corresponding to the incident light, displaying a spatial floating video that is a real image based on the video displayed on the display device 1.

[0024] The resolution of this floating image in space depends greatly on the outer shape D and pitch P of the retroreflective portion of the retroreflective plate 2 shown in Fig. 2A(2), in addition to the resolution of the liquid crystal display panel 11. For example, when using a 7-inch WUXGA (1920×1200 pixels) liquid crystal display panel, even if one pixel (one triplet) is about 80 μm, if the diameter D of the retroreflective portion is 240 μm and the pitch is 300 μm, for example, one pixel of the floating image in space will be equivalent to 300 μm. Therefore, the effective resolution of the floating video is reduced to about 1 / 3.

[0025] Therefore, in order to make the resolution of the floating video in space equivalent to the resolution of the display device 1, it is desirable to make the diameter and pitch of the retroreflective portion closer to one pixel of the liquid crystal display panel. On the other hand, in order to suppress the occurrence of moiré due to the pixels of the retroreflective plate and the liquid crystal display panel, it is advisable to design by deviating the respective pitch ratios from an integer multiple of one pixel. Also, the shape may be arranged so that any side of the retroreflective portion does not overlap with any side of one pixel of the liquid crystal display panel.

[0026] Note that the surface shape of the retroreflective plate according to this embodiment is not limited to the above example. It may have various surface shapes that realize retroreflection. Specifically, a retroreflective element in which triangular pyramid prisms, hexagonal pyramid prisms, other polygonal prisms, multi-vertex prisms, or a combination thereof are periodically arranged may be provided on the surface of the retroreflective plate of this embodiment. Or, a retroreflective element that forms a cube corner by periodically arranging these prisms may be provided on the surface of the retroreflective plate of this embodiment. These can also be expressed as a corner reflector array or a multi-faceted reflector array. Or, a capsule lens type retroreflective element in which glass beads are periodically arranged may be provided on the surface of the retroreflective plate of this embodiment. Since the detailed configuration of these retroreflective elements may use existing technologies, detailed description is omitted. Specifically, the technologies disclosed in JP-A-2001-33609, JP-A-2001-264525, JP-A-2005-181555, JP-A-2008-70898, JP-A-2009-229942, etc. may be used.

[0027] <Another Configuration Example 1 of the Optical System of the Spatial Floating Image Display Device> Another configuration example of the optical system of the spatial floating image display device will be described with reference to FIG. 2B. In FIG. 2B, components labeled with the same reference numerals as in FIG. 2A have the same functions and configurations as those in FIG. 2A. For the sake of simplicity, repeated explanations of such components will be omitted.

[0028] In the optical system of FIG. 2B, similar to FIG. 2A, image light of a specific polarization is output from the display device 1. The image light of the specific polarization output from the display device 1 is input to the polarization separation member 101B. The polarization separation member 101B is a member that selectively transmits the image light of the specific polarization. Different from the polarization separation member 101 in FIG. 2A, the polarization separation member 101B is not integrated with the transparent member 100 and has an independent plate-like shape. Therefore, the polarization separation member 101B may be referred to as a polarization separation plate. The polarization separation member 101B may be configured, for example, as a reflective polarizing plate formed by attaching a polarization separation sheet to a transparent member. Or it may be formed of a metal multilayer film or the like that selectively transmits a specific polarization of the transparent member and reflects the polarization of other specific polarizations. In FIG. 2B, the polarization separation member 101B is configured to transmit the image light of the specific polarization output from the display device 1.

[0029] The image light that has passed through the polarization separation member 101B is incident on the retroreflector 2. A λ / 4 plate 21 is provided on the image light incident surface of the retroreflector. The image light is polarization-converted from a specific polarization to the other polarization by passing through the λ / 4 plate 21 twice, once when entering the retroreflector and once when exiting. Here, since the polarization separation member 101B has the property of reflecting the polarization of the other polarization that has been polarization-converted by the λ / 4 plate 21, the image light after polarization conversion is reflected by the polarization separation member 101B. The image light reflected by the polarization separation member 101B passes through the transparent member 100 and forms a spatial floating image 3, which is a real image, outside the transparent member 100.

[0030] Here, a first example of the polarization design in the optical system of FIG. 2B will be described. For example, the configuration may be such that P-polarized video light is emitted from the display device 1 to the polarization separation member 101B, and the polarization separation member 101B has the characteristic of reflecting S-polarized light and transmitting P-polarized light. In this case, the P-polarized video light that has reached the polarization separation member 101B from the display device 1 passes through the polarization separation member 101B and travels toward the retroreflector 2. When the video light is reflected by the retroreflector 2, since it passes through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2 twice, the video light is converted from P-polarized light to S-polarized light. The video light converted to S-polarized light travels toward the polarization separation member 101B again. Here, since the polarization separation member 101B has the characteristic of reflecting S-polarized light and transmitting P-polarized light, the S-polarized video light is reflected by the polarization separation member 101 and passes through the transparent member 100. Since the video light that has passed through the transparent member 100 is the light generated by the retroreflector 2, a spatial floating image 3, which is an optical image of the display image of the display device 1, is formed at a position that has a mirror image relationship with the display image of the display device 1 with respect to the polarization separation member 101B. Such a polarization design can preferably form the spatial floating image 3.

[0031] Next, a second example of the polarization design in the optical system of FIG. 2B will be described. For example, it may be configured such that S-polarized video light is emitted from the display device 1 to the polarization separation member 101B, and the polarization separation member 101B has the property of reflecting P-polarized light and transmitting S-polarized light. In this case, the S-polarized video light that has reached the polarization separation member 101B from the display device 1 passes through the polarization separation member 101B and travels toward the retroreflector 2. When the video light is reflected by the retroreflector 2, since it passes through the λ / 4 plate 21 provided on the incident surface of the retroreflector 2 twice, the video light is converted from S-polarized light to P-polarized light. The video light converted to P-polarized light travels again toward the polarization separation member 101B. Here, since the polarization separation member 101B has the property of reflecting P-polarized light and transmitting S-polarized light, the P-polarized video light is reflected by the polarization separation member 101 and passes through the transparent member 100. Since the video light that has passed through the transparent member 100 is the light generated by the retroreflector 2, a spatial floating image 3, which is an optical image of the display image of the display device 1, is formed at a position that has a mirror relationship with the display image of the display device 1 with respect to the polarization separation member 101B. Such a polarization design can preferably form the spatial floating image 3.

[0032] In Fig. 2B, the image display surface of the display device 1 and the surface of the retroreflective plate 2 are arranged in parallel. The polarization separation member 101B is arranged at an angle α (for example, 30°) with respect to the image display surface of the display device 1 and the surface of the retroreflective plate 2. Then, in the reflection of the polarization separation member 101B, the traveling direction of the image light reflected by the polarization separation member 101B (the direction of the principal ray of the image light) is different from the traveling direction of the image light incident from the retroreflective plate 2 (the direction of the principal ray of the image light) by an angle β (for example, 60°). By configuring in this way, in the optical system of Fig. 2B, the image light is output at a predetermined angle shown toward the outside of the transparent member 100, and the spatial floating image 3 which is a real image is formed. In the configuration of Fig. 2B, when viewed by the user from the direction of arrow A, the spatial floating image 3 is viewed as a bright image. However, when viewed by another person from the direction of arrow B, the spatial floating image 3 cannot be viewed as an image at all. This characteristic is very suitable for use in a system for displaying an image that requires high security or an image with high confidentiality that needs to be concealed from a person facing the user.

[0033] As described above, in the optical system of Fig. 2B, although it is an optical system with a different configuration from the optical system of Fig. 2A, it is possible to form a suitable spatial floating image in the same manner as the optical system of Fig. 2A.

[0034] An absorption type polarizing plate may be provided on the surface of the transparent member 100 on the side of the polarization separation member 101B. The absorption type polarizing plate may be an absorption type polarizing plate that transmits the polarization wave of the image light from the polarization separation member 101B and absorbs the polarization wave whose phase is 90° different from the polarization wave of the image light from the polarization separation member 101B. In this way, while allowing sufficient transmission of the image light for forming the spatial floating image 3, it is possible to reduce the external light incident from the side of the spatial floating image 3 of the transparent member 100 by about 50%. Thereby, it is possible to reduce the stray light in the optical system of Fig. 2B based on the external light incident from the side of the spatial floating image 3 of the transparent member 100.

[0035] <Another configuration example 2 of the optical system of the spatial floating image display device> Another configuration example of the optical system of the spatial floating image display device will be described with reference to FIG. 2C. In FIG. 2C, components labeled with the same reference numerals as in FIG. 2B have the same functions and configurations as those in FIG. 2B. For such components, repetitive descriptions will be omitted for simplicity.

[0036] The difference between the optical system of FIG. 2C and the optical system of FIG. 2B is only the arrangement angle of the polarization separation member 101B with respect to the image display surface of the display device 1 and the surface of the retroreflective plate 2. Since all other configurations are the same as those of the optical system of FIG. 2B, repetitive descriptions will be omitted. Since the polarization design of the optical system of FIG. 2C is the same as that of the optical system of FIG. 2B, repetitive descriptions will be omitted.

[0037] In the optical system of FIG. 2C, the polarization separation member 101B is arranged at an angle α with respect to the image display surface of the display device 1 and the surface of the retroreflective plate 2. In FIG. 2C, the angle α is 45°. With this configuration, in the reflection of the polarization separation member 101B, the angle β formed between the traveling direction of the image light incident from the retroreflective plate 2 (the direction of the principal ray of the image light) and the traveling direction of the image light reflected by the polarization separation member 101B (the direction of the principal ray of the image light) is 90°. With this configuration, the image display surface of the display device 1 and the surface of the retroreflective plate 2 are perpendicular to the traveling direction of the image light reflected by the polarization separation member 101B, and the angular relationship of the surfaces constituting the optical system can be simplified. If the surface of the transparent member 100 is arranged perpendicular to the traveling direction of the image light reflected by the polarization separation member 101B, the angular relationship of the surfaces constituting the optical system can be further simplified. In the configuration of FIG. 2C, when viewed by the user from the direction of arrow A, the spatial floating image 3 is viewed as a bright image. However, when viewed by another person from the direction of arrow B, the spatial floating image 3 cannot be viewed as an image at all. This characteristic is very suitable for use in systems that display images requiring high security or highly confidential images that need to be concealed from people facing the user.

[0038] As described above, the optical system in FIG. 2C is an optical system having a configuration different from those of the optical systems in FIGS. 2A and 2B, but like the optical systems in FIGS. 2A and 2B, it can form a suitable aerial image. Also, the angles of the surfaces constituting the optical system can be made simpler.

[0039] Note that an absorption type polarizing plate may be provided on the surface of the transparent member 100 on the side of the polarization separation member 101B. The absorption type polarizing plate may be an absorption type polarizing plate that transmits the polarization wave of the video light from the polarization separation member 101B and absorbs the polarization wave having a phase difference of 90° from the polarization wave of the video light from the polarization separation member 101B. In this way, while allowing sufficient transmission of the video light for forming the aerial image 3, the external light incident from the side of the aerial image 3 of the transparent member 100 can be reduced by about 50%. Thereby, the stray light in the optical system of FIG. 2C based on the external light incident from the side of the aerial image 3 of the transparent member 100 can be reduced.

[0040] <Another Configuration Example 3 of the Optical System of the Aerial Image Display Device> Another configuration example of the optical system of the aerial image display device will be described with reference to FIG. 2D. The optical system of FIG. 2D is an optical system using a retroreflector 5, which is different from the retroreflector 2 used in FIGS. 2A to 2C. Hereinafter, with reference to FIGS. 2D to 2I, another configuration example 3 of the optical system will be described more specifically. In FIG. 2D, the components denoted by the same reference numerals as those in FIGS. 2A to 2C have the same functions and configurations as those in FIGS. 2A to 2C. For such components, repetitive description will not be given in order to simplify the explanation.

[0041] FIG. 2D is a diagram showing an example of the main configuration and the retroreflective portion configuration of an aerial image display device according to an embodiment of the present invention. A display device 1 that emits video light is provided in an oblique direction of a transparent member 100 such as glass. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates light.

[0042] The chief ray 9020, which represents the light beam emitted from the display device 1, travels toward the retroreflector 5 and is incident on the retroreflector 5 at an incident angle α. The incident angle α may be, for example, 45°. However, the incident angle α is not limited to 45°, and it can also be used, for example, within the range of 45° ± 15°.

[0043] The retroreflector 5 is an optical member having an optical property of retroreflecting at least part of the light rays in some directions. Also, since the reflected light rays have the optical property of forming an image, the retroreflector 5 may be referred to as an imaging optical member or an imaging optical plate.

[0044] Regarding the specific configuration of the retroreflector 5, it will be described in detail with reference to FIGS. 2E, 2F, etc. By the retroreflector 5, the chief ray 9020 is retroreflected in the x and y directions while traveling in the z direction. As a result, the reflected ray 9021 travels in a direction away from the retroreflector 5 along an optical path that is mirror-symmetric with respect to the chief ray 9020 with the retroreflector 5 as the reference, passes through the transparent member 100, and forms a spatial floating image 3 as a real image on the imaging surface.

[0045] The light beam that forms the spatial floating image 3 is a collection of light rays that converge from the retroreflector 5 to the optical image of the spatial floating image 3, and these light rays continue to travel straight even after passing through the optical image of the spatial floating image 3. Therefore, unlike a diffused image formed on a screen by a general projector or the like, the spatial floating image 3 is an image with high directivity. Thus, in the configuration of FIG. 2, when the user views from the direction of arrow A, the spatial floating image 3 is viewed as a bright image. However, when another person views from the direction of arrow B, the spatial floating image 3 cannot be viewed as an image at all. This characteristic is suitable for use in systems that display images requiring high security or highly confidential images that need to be concealed from people facing the user.

[0046] An example of the configuration of the retroreflector 5 will be described with reference to FIGS. 2E and 2F. The retroreflector 5 has a configuration in which a plurality of corner reflectors 9040 are arranged in an array on the surface of a transparent member. This may be called a corner reflector array or a polyhedral reflector array. The specific configuration of the corner reflector 9040 will be described in detail with reference to FIGS. 2G, 2H, and 2I. The light rays 9111, 9112, 9113, and 9114 emitted from the light source 9110 are reflected twice by the two mirror surfaces 9041 and 9042 of the corner reflector 9040, becoming the reflected light rays 9121, 9122, 9123, and 9124. This double reflection results in a retroreflective reflection that folds back in the same direction as the incident direction (advances in the direction rotated by 180°) with respect to the x and y directions, and a regular reflection in which the incident angle and the reflection angle coincide due to total reflection with respect to the z direction.

[0047] That is, the light rays 9111 to 9114 generate the reflected light rays 9121 to 9124 on a straight line symmetric with respect to the z direction with respect to the corner reflector 9040, forming a virtual image 9120 in the air. The light rays 9111 to 9114 emitted from the light source 9110 are four light rays representing the diffused light from the light source 9110. Depending on the diffusion characteristics of the light source 9110, the light rays incident on the retroreflector 5 are not limited to these, but any incident light ray causes a similar reflection and forms a virtual image 9120 in the air. For ease of viewing the drawing, the position of the light source 9110 and the position of the virtual image 9120 in the x direction are shifted in the drawing, but actually the position of the light source 9110 and the position of the virtual image 9120 in the x direction are the same position, and they overlap when viewed from the z direction.

[0048] Next, the configuration and effect of the corner reflector 9040 that constitutes the retroreflector 5 will be described with reference to FIGS. 2G, 2H, and 2I. The corner reflector 9040 is a rectangular parallelepiped in which only two specific surfaces are the mirror surfaces 9041 and 9042, and the other four surfaces are formed of a transparent member. The retroreflector 5 has a configuration in which the corner reflectors 9040 are arranged in an array such that their corresponding mirror surfaces face the same direction.

[0049] When viewed from above (+z direction), the light ray 9111 emitted from the light source 9110 is incident on the mirror surface 9041 (or mirror surface 9042) at a specific incident angle, and after total reflection at the reflection point 9130, it is totally reflected again at the reflection point 9132 on the mirror surface 9042 (or mirror surface 9041).

[0050] Assuming the incident angle of the light ray 9111 with respect to the mirror surface 9041 (or mirror surface 9042) is θ, the incident angle of the first reflected light ray 9131 reflected by the mirror surface 9041 (or mirror surface 9042) with respect to the mirror surface 9042 (or mirror surface 9041) can be expressed as 90° - θ. Therefore, for the light ray 9111, the second reflected light ray 9121 obtains a rotation of 2θ by the first reflection and 2×(90° - θ) by the second reflection, resulting in a total reverse optical path of 180°. On the other hand, when viewed from the side (the intermediate direction between -x and -y), total reflection with respect to the z direction occurs only once. Therefore, assuming the incident angle with respect to the mirror surface 9041 or mirror surface 9042 is φ, for the light ray 9111, the reflected light ray 9121 obtains a rotation of 2×φ by one reflection.

[0051] From the above, the light rays incident on the corner reflector 9040 produce a recursive reflection with a reverse optical path in the x and y directions, and a regular reflection by total reflection in the z direction. Considering the retroreflective plate 5, similar reflections are caused in each optical path. Therefore, in the x and y directions, an inverted optical path with convergence forms an image at a point symmetric with respect to the z-axis direction.

[0052] Here, in the optical systems of FIGS. 2A to 2C, the retroreflective plate 2 has a recursive reflection characteristic in three axial directions. As a result, when a diffusive incident light beam is incident on the retroreflective plate 2, a convergent reflected light beam travels toward the side where the light source of the incident light ray exists with respect to the retroreflective plate 2. The convergent reflected light beam forms an image in the air to form a spatial floating image 3. The traveling direction of the principal ray of the convergent reflected light beam reflected from the retroreflective plate 2 is opposite to the traveling direction of the principal ray of the diffusive incident light beam incident on the retroreflective plate 2.

[0053] In contrast, in the optical system of FIG. 2D, the retroreflective plate 5 has retroreflective characteristics in two axial directions and specularly reflects in the other one axial direction. Thus, when a diffusive incident light beam is incident on the retroreflective plate 5, the convergent reflected light beam reflected by the corner reflector array travels toward the side opposite to the side where the light source of the incident light ray exists with respect to the retroreflective plate 5. The convergent reflected light beam forms a virtual image in the air and forms the spatial floating image 3.

[0054] The traveling direction of the principal ray of the convergent reflected light beam reflected by the corner reflector array of the retroreflective plate 5 does not become the reverse direction of the traveling direction of the principal ray of the diffusive incident light beam incident on the retroreflective plate 5. The normal direction component of the plate-shaped surface of the retroreflective plate 5 with respect to the traveling direction of the principal ray of the diffusive incident light beam incident on the retroreflective plate 5 and the normal direction component of the plate-shaped surface of the retroreflective plate 5 with respect to the traveling direction of the principal ray after the light beam is reflected by the retroreflective plate 5 and becomes a convergent reflected light beam travel straight without changing before and after reflection by the corner reflector array.

[0055] That is, due to the reflection in the retroreflective plate 5, the diffusive incident light beam is converted into a convergent reflected light beam, but in the normal direction of the plate-shaped surface of the retroreflective plate 5, the light beam travels through the retroreflective plate 5. Here, the diffusive incident light beam incident on the retroreflective plate 5 and the convergent reflected light beam emitted from the retroreflective plate 5 have a geometrically plane-symmetric relationship with respect to the plate-shaped surface of the retroreflective plate 5.

[0056] The resolution of the spatial floating image formed by the light rays from the display device 1 depends greatly on the diameter D and pitch P (not shown) of the retroreflective portion of the retroreflective plate 5 shown in FIGS. 2E and 2F, in addition to the resolution of the liquid crystal display panel 11. For example, when using a 7-inch WUXGA (1920×1200 pixels) liquid crystal display panel, even if one pixel (one triplet) is about 80 μm, if the diameter D of the retroreflective portion is 240 μm and the pitch P is 300 μm, for example, one pixel of the spatial floating image corresponds to 300 μm. Therefore, the effective resolution of the spatial floating image is reduced to about one-third.

[0057] Therefore, in order to make the resolution of the spatially floating image equivalent to that of the display device 1, it is desirable to make the diameter D and pitch P of the retroreflective portion approach one pixel of the liquid crystal display panel. On the other hand, in order to suppress the occurrence of moiré due to the retroreflective plate and the pixels of the liquid crystal display panel, it is advisable to design by deviating the respective pitch ratios from an integral multiple of one pixel. Also, the shape may be arranged so that none of the sides of the retroreflective portion overlap any of the sides of one pixel of the liquid crystal display panel.

[0058] Note that the shape of the retroreflective plate (imaging optical plate) according to this embodiment is not limited to the above example. It may have various shapes that realize retroreflection. Specifically, various cubic corner bodies, corner reflector arrays may be used, or a slit mirror array, a two-sided corner reflector array, a multi-sided reflector array, or a shape in which combinations of its reflecting surfaces are periodically arranged may be used. Alternatively, a capsule lens type retroreflective element in which glass beads are periodically arranged may be provided on the surface of the retroreflective plate of this embodiment. Since the detailed configurations of these retroreflective elements may use existing technologies, detailed descriptions thereof are omitted. Specifically, the technologies disclosed in JP-A-2017-33005, JP-A-2019-133110, JP-A-2017-67933, WO2009 / 131128, etc. may be used.

[0059] In the optical system of Fig. 2D, the video light emitted from the display device 1 may be in any polarization state. There is no problem whether it is S-polarized light or P-polarized light.

[0060] As described above, in the optical system of Fig. 2D, although it is an optical system using a retroreflective plate different from the optical systems of Figs. 2A to 2C, a more suitable spatially floating image can be formed in the same manner as the optical systems of Figs. 2A to 2C.

[0061] According to the optical systems of Figs. 2A, 2B, 2C, and 2D described above, a brighter and higher-quality spatially floating image can be provided.

[0062] <<Block diagram of the internal configuration of the spatial floating image display device>> Next, a block diagram of the internal configuration of the spatial floating image display device 1000 will be described. FIG. 3 is a block diagram showing an example of the internal configuration of the spatial floating image display device 1000.

[0063] The spatial floating image display device 1000 includes a retroreflective portion 1101, an image display portion 1102, a light guide 1104, a light source 1105, a power supply 1106, an external power input interface 1111, an operation input portion 1107, a nonvolatile memory 1108, a memory 1109, a control portion 1110, an image signal input portion 1131, an audio signal input portion 1133, a communication portion 1132, an air operation detection sensor 1351, an air operation detection portion 1350, an audio output portion 1140, a microphone 1139, an image control portion 1160, a storage portion 1170, an imaging portion 1180, etc. Note that a removable media interface 1134, an attitude sensor 1113, a transmissive self-emitting image display device 1650, a second display device 1680, or a secondary battery 1112, etc. may be provided.

[0064] Each component of the spatial floating image display device 1000 is arranged in a housing 1190. Note that the imaging portion 1180 and the air operation detection sensor 1351 shown in FIG. 3 may be provided outside the housing 1190.

[0065] The retroreflective portion 1101 in FIG. 3 corresponds to the retroreflective plate 2 in FIGS. 2A, 2B, and 2C. The retroreflective portion 1101 retroreflects the light modulated by the image display portion 1102. Among the reflected light from the retroreflective portion 1101, a spatial floating image 3 is formed by the light output to the outside of the spatial floating image display device 1000. When the optical system of FIG. 2D is applied, the retroreflective portion 1101 corresponds to the retroreflective plate 5 in FIG. 2D.

[0066] The video display unit 1102 in FIG. 3 corresponds to the liquid crystal display panel 11 in FIGS. 2A, 2B, and 2C. The light source 1105 in FIG. 3 corresponds to the light source device 13 in FIGS. 2A, 2B, and 2C. And the video display unit 1102, the light guide 1104, and the light source 1105 in FIG. 3 correspond to the display device 1 in FIGS. 2A, 2B, and 2C.

[0067] The video display unit 1102 is a display unit that modulates the transmitted light to generate a video based on the video signal input under the control of a video control unit 1160 described later. As the video display unit 1102 (the above-described liquid crystal display panel 11), for example, a transmissive liquid crystal panel is used, but it is not limited thereto. Also, as the video display unit 1102, for example, a reflective liquid crystal panel or a DMD (Digital Micromirror Device: registered trademark) panel that modulates the reflected light may be used.

[0068] The light source 1105 generates light for the video display unit 1102 and is a solid light source such as an LED light source (LED: Light Emitting Diode) or a laser light source. The power supply 1106 converts the AC current input from the outside through the external power supply input interface 1111 into a DC current and supplies power to the light source 1105. Also, the power supply 1106 supplies the necessary DC current to each part in the spatial floating video display device 1000. The secondary battery 1112 stores the power supplied from the power supply 1106. Also, the secondary battery 1112 supplies power to the light source 1105 and other components that require power when no power is supplied from the outside through the external power supply input interface 1111. That is, when the spatial floating video display device 1000 includes the secondary battery 1112, the user can use the spatial floating video display device 1000 even when no power is supplied from the outside.

[0069] The light guide 1104 guides the light generated by the light source 1105 and irradiates the image display unit 1102. The combination of the light guide 1104 and the light source 1105 can also be referred to as the backlight of the image display unit 1102. The light guide 1104 may mainly be configured using glass. The light guide 1104 may mainly be configured using plastic. The light guide 1104 may be configured using a mirror. Various methods can be considered for the combination of the light guide 1104 and the light source 1105. Specific configuration examples of the combination of the light guide 1104 and the light source 1105 will be described in detail later.

[0070] The air operation detection sensor 1351 is a sensor that detects the operation of the spatially floating image 3 by an operating object such as a user's finger. The air operation detection sensor 1351 senses, for example, a range that overlaps with the entire display range of the spatially floating image 3. Note that the air operation detection sensor 1351 may sense only a range that overlaps with at least a part of the display range of the spatially floating image 3.

[0071] Specific examples of the air operation detection sensor 1351 include distance sensors using non-visible light such as infrared rays, non-visible light lasers, ultrasonic waves, etc. Also, the air operation detection sensor 1351 may be configured by combining a plurality of sensors in a plurality of ways so as to be able to detect two-dimensional plane coordinates. Also, the air operation detection sensor 1351 may be configured by ToF (Time of Flight) type LiDAR (Light Detection and Ranging) or an image sensor.

[0072] The air operation detection sensor 1351 only needs to be able to sense a touch operation or the like by the user's finger on the object displayed as the spatially floating image 3. Such sensing can be performed using existing technologies.

[0073] The air operation detection unit 1350 acquires a sensing signal from the air operation detection sensor 1351, and based on the sensing signal, determines whether there is contact with an object of the spatial floating video 3 by the user's finger, calculates the position (contact position) where the user's finger and the object are in contact, and so on. The air operation detection unit 1350 is composed of a circuit such as an FPGA (Field Programmable Gate Array), for example. Also, some functions of the air operation detection unit 1350 may be realized by software using a spatial operation detection program executed by, for example, the control unit 1110 or the video control unit 1160. The air operation detection sensor 1351 and the air operation detection unit 1350 may be integrally configured. The air operation detection unit 1350 and the control unit 1110 or the video control unit 1160 may be integrally configured.

[0074] The air operation detection sensor 1351 and the air operation detection unit 1350 may be built into the spatial floating video display device 1000, or may be provided externally as a separate unit from the spatial floating video display device 1000. When provided as a separate unit from the spatial floating video display device 1000, the air operation detection sensor 1351 and the air operation detection unit 1350 are configured to be able to transmit information and signals to the spatial floating video display device 1000 via a wired or wireless communication connection path or a video signal transmission path. Thereby, it is possible to construct a system in which the spatial floating video display device 1000 without an air operation detection function is used as the main body, and only the air operation detection function can be added as an option.

[0075] Also, only the air operation detection sensor 1351 may be a separate unit, and the air operation detection unit 1350 may be built into the spatial floating video display device 1000. When it is desired to arrange the air operation detection sensor 1351 more freely with respect to the installation position of the spatial floating video display device 1000, etc., there are advantages to a configuration in which only the air operation detection sensor 1351 is a separate unit.

[0076] The imaging unit 1180 is a camera having an image sensor, and images the space near the spatial floating image 3 and / or the face, arm, finger, etc. of the user 230. A plurality of imaging units 1180 may be provided. For example, the imaging unit 1180 may be provided as a stereo camera. By using a plurality of imaging units 1180 or by using an imaging unit with a depth sensor, the air operation detection unit 1350 can be assisted during the detection process of the touch operation on the spatial floating image 3 by the user 230. The imaging unit 1180 may be provided separately from the spatial floating image display device 1000. When the imaging unit 1180 is provided separately from the spatial floating image display device 1000, it may be configured so that an imaging signal can be transmitted to the spatial floating image display device 1000 via a wired or wireless communication connection path or the like.

[0077] For example, when the air operation detection sensor 1351 is configured as an object intrusion sensor that detects the presence or absence of an object intrusion into a plane (intrusion detection plane) including the display surface (display range) of the spatial floating image 3, information such as how far an object (for example, a user's finger) that has not intruded into the intrusion detection plane is from the intrusion detection plane or how close the object is to the intrusion detection plane may not be detected by the air operation detection sensor 1351.

[0078] In such a case, by using information such as the object depth calculation information based on the captured images of the plurality of imaging units 1180 and the object depth information by the depth sensor, the distance between the object and the intrusion detection plane (spatial floating image 3) can be calculated. Then, various information such as these depth calculation information, depth information, and the distance between the object and the intrusion detection plane is used for various display controls for the spatial floating image 3.

[0079] Further, the air operation detection unit 1350 may detect the touch operation on the spatial floating image 3 by the user 230 based on the captured image of the imaging unit 1180 without using the air operation detection sensor 1351. In this case, the imaging unit 1180 may be referred to as an air operation detection sensor.

[0080] Further, the imaging unit 1180 may image the face of the user who operates the spatial floating image 3, and the control unit 1110 or the like may perform user identification processing. Further, in order to determine whether another person is standing around or behind the user who operates the spatial floating image 3 and whether the other person is peeping at the user's operation on the spatial floating image 3, the imaging unit 1180 may image a range including the user who operates the spatial floating image 3 and the peripheral area of the user.

[0081] The operation input unit 1107 is, for example, an operation button, a signal reception unit such as a remote controller, or an infrared light receiving unit, and inputs a signal for an operation different from the air operation (touch operation) by the user. Separately from the above-mentioned user who performs a touch operation on the spatial floating image 3, the operation input unit 1107 may be used, for example, by an administrator to operate the spatial floating image display device 1000.

[0082] The video signal input unit 1131 connects to an external video output device and inputs video data (video signal). Various digital video input interfaces can be considered for the video signal input unit 1131. For example, it may be configured with a video input interface of the HDMI (registered trademark) (High-Definition Multimedia Interface) standard, a video input interface of the DVI (Digital Visual Interface) standard, or a video input interface of the DisplayPort standard. Alternatively, an analog video input interface such as analog RGB or composite video may be provided.

[0083] The audio signal input unit 1133 connects to an external audio output device and inputs audio data (audio signal). The audio signal input unit 1133 may be configured with an audio input interface of the HDMI standard, an optical digital terminal interface, or a coaxial digital terminal interface, etc. In the case of an interface of the HDMI standard, the video signal input unit 1131 and the audio signal input unit 1133 may be configured as an integrated interface with integrated terminals and cables.

[0084] The voice output unit 1140 can output voice based on the voice data input to the voice signal input unit 1133. The voice output unit 1140 may be composed of a speaker 1140. The voice output unit 1140 may be provided with a part for performing voice synthesis processing or the like. Further, the voice output unit 1140 may output built-in operation sounds or error warning sounds. Alternatively, a configuration that outputs a digital signal to an external device, such as the Audio Return Channel function defined in the HDMI standard, may be used as the voice output unit 1140.

[0085] The voice input unit 1139 may be composed of a microphone 1139. The microphone 1139 is a microphone that picks up sounds around the spatial floating video display device 1000, converts them into signals, and generates voice signals. The microphone may record the voice of a person such as the user's voice, and the generated voice signal may be subjected to voice recognition processing by the control unit 1110 or the like to obtain character information from the voice signal. The voice input unit 1139 may be provided with a part for performing voice recognition processing or the like. Note that the voice output unit 1140, the voice input unit 1139, etc. may be connected as external devices of the spatial floating video display device 1000.

[0086] The non-volatile memory 1108 stores various data used in the spatial floating video display device 1000. The data stored in the non-volatile memory 1108 includes, for example, various operation data for display on the spatial floating video 3, display icons, data of objects for the user to operate, layout information, and the like. The memory 1109 stores video data to be displayed as the spatial floating video 3, control data of the device, and the like.

[0087] The control unit 1110 includes a processor and controls the operations of the connected units. Further, the control unit 1110 may perform arithmetic processing based on the information obtained from each unit within the spatial floating video display device 1000 in cooperation with the program stored in the memory 1109.

[0088] The communication unit 1132 communicates with external devices, external servers, etc. via a wired or wireless communication interface. When the communication unit 1132 has a wired communication interface, the wired communication interface may be constituted by, for example, a LAN interface conforming to the Ethernet standard. When the communication unit 1132 has a wireless communication interface, it may be constituted by, for example, a communication interface of the Wi-Fi system, a communication interface of the Bluetooth system, a mobile communication interface such as 4G or 5G. Through the communication via the communication unit 1132, various data such as video data, image data, and audio data are transmitted and received.

[0089] Also, the removable media interface 1134 is an interface for connecting a removable recording medium (removable media). The removable recording medium (removable media) may be constituted by a semiconductor element memory such as a solid state drive (SSD), a magnetic recording medium recording device such as a hard disk drive (HDD), or an optical recording medium such as an optical disk. The removable media interface 1134 can read various information such as various data such as video data, image data, and audio data recorded on the removable recording medium. The video data, image data, etc. recorded on the removable recording medium are output as the spatial floating image 3 via the video display unit 1102 and the retroreflective unit 1101.

[0090] The storage unit 1170 is a storage device for recording various information such as various data such as video data, image data, and audio data. The storage unit 1170 may be constituted by a magnetic recording medium recording device such as a hard disk drive (HDD) or a semiconductor element memory such as a solid state drive (SSD). In the storage unit 1170, for example, various information such as various data such as video data, image data, and audio data may be recorded in advance at the time of product shipment. Also, the storage unit 1170 may record various information such as various data such as video data, image data, and audio data acquired from external devices or external servers via the communication unit 1132.

[0091] The video data, image data, etc. recorded in the storage unit 1170 are output as a spatial floating image 3 via the video display unit 1102 and the retroreflective unit 1101 based on the processing by the video control unit 1160. The video data, image data, etc. such as display icons and objects for the user to operate, which are displayed as the spatial floating image 3, are also recorded in the storage unit 1170. The layout information of the display icons, objects, etc. displayed as the spatial floating image 3, and the information of various metadata related to the objects are also recorded in the storage unit 1170.

[0092] The audio data recorded in the storage unit 1170 is output as audio from, for example, the audio output unit 1140.

[0093] The video control unit 1160 performs various controls on the video signal input to the video display unit 1102. Based on the video signal (video data), the video control unit 1160 creates a video signal (display data) for displaying a video on the video display unit 1102 (for example, the liquid crystal display panel 11 of the aforementioned display device 1) and supplies it to the video display unit 1102. The video control unit 1160 may also be referred to as a video processing circuit and may be composed of hardware such as an ASIC, an FPGA, or a video processor. Note that the video control unit 1160 may also be referred to as a video processing unit or an image processing unit. The video control unit 1160 performs control such as video switching, for example, which video signal among the video signals stored in the memory 1109 and the video signals (video data) input to the video signal input unit 1131 is to be input to the video display unit 1102.

[0094] Note that the control unit 1110 may perform the same processing as the video control unit 1160. In that case, the control unit 1110 may also be referred to as a video processing unit or the like. At least one of the control unit 1110, the video control unit 1160, the air operation detection unit 1360, etc. may perform specific control processing. In that case, the control unit 1110, the video control unit 1160, the air operation detection unit 1360, etc. may also be referred to as a video processing unit.

[0095] Further, the video control unit 1160 may generate a superimposed video signal obtained by superimposing the video signal stored in the memory 1109 and the video signal input from the video signal input unit 1131, and form a composite video as the spatial floating video 3 by inputting the superimposed video signal to the video display unit 1102.

[0096] Further, the video control unit 1160 may perform control to perform image processing on the video signal input from the video signal input unit 1131, the video signal stored in the memory 1109, etc. Examples of the image processing include scaling processing such as enlarging, reducing, and deforming an image, brightness adjustment processing for changing brightness, contrast adjustment processing for changing the contrast curve of an image, and Retinex processing for decomposing an image into light components and changing the weighting for each component.

[0097] Further, the video control unit 1160 may perform special effect video processing or the like for assisting the user's in-air operation (touch operation) on the video signal input to the video display unit 1102. The special effect video processing is performed based on, for example, the detection result of the user's touch operation by the in-air operation detection unit 1350 or the captured image of the user by the imaging unit 1180. Also, the video control unit 1160 or the like may perform audio control processing when outputting audio from the audio output unit 1140 simultaneously with the spatial floating video 3. An audio control unit for the audio control processing may be provided separately from the video control unit 1160.

[0098] The posture sensor 1113 is a sensor composed of a gravity sensor, an acceleration sensor, or a combination thereof, and can detect the posture in which the spatial floating image display device 1000 is installed. Based on the posture detection result of the posture sensor 1113, the control unit 1110 may control the operations of the respective connected components. For example, when an unfavorable posture as the user's usage state is detected, control may be performed to stop the display of the image being displayed on the video display unit 1102 and display an error message to the user. Alternatively, when the posture sensor 1113 detects that the installation posture of the spatial floating image display device 1000 has changed, control may be performed to rotate the orientation of the image being displayed on the video display unit 1102.

[0099] As described so far, the spatial floating image display device 1000 is equipped with various functions. However, the spatial floating image display device 1000 does not necessarily need to have all of these functions, and any configuration may be used as long as it has the function of forming the spatial floating image 3.

[0100] <Configuration Example of Spatial Floating Image Display Device> Next, a configuration example of the spatial floating image display device will be described. The layout of the components of the spatial floating image display device according to this embodiment can have various layouts depending on the usage form. Hereinafter, each layout in FIGS. 4A to 4P will be described. In any of the examples in FIGS. 4A to 4P, the thick line surrounding the components (such as the display device 1) of the spatial floating image display device 1000 shows an example of the housing structure (housing 1190 in FIG. 3) of the spatial floating image display device 1000.

[0101] FIG. 4A is a diagram showing an example of the configuration of a spatial floating image display device. The spatial floating image display device 1000 shown in FIG. 4A is equipped with an optical system corresponding to the optical system of FIG. 2A. In the spatial floating image display device 1000 shown in FIG. 4A, it is installed horizontally so that the surface on which the spatial floating image 3 is formed faces upward. That is, in FIG. 4A, for the spatial floating image display device 1000, a transparent member 100 is installed on the upper surface of the device. The spatial floating image 3 is formed above the surface of the transparent member 100 of the spatial floating image display device 1000. The light of the spatial floating image 3 travels in an obliquely upward direction. When the air operation detection sensor 1351 is provided as shown in the figure, the operation of the spatial floating image 3 by the finger of the user 230 can be detected. Here, the x direction is the left - right direction as seen from the user, the y direction is the front - back direction (depth direction) as seen from the user, and the z direction is the up - down direction (vertical direction). Hereinafter, since the definitions of the x direction, y direction, and z direction in each figure of FIG. 4 are the same, repeated explanations will be omitted.

[0102] FIG. 4B is a diagram showing an example of the configuration of a spatial floating image display device. The spatial floating image display device 1000 shown in FIG. 4B is equipped with an optical system corresponding to the optical system of FIG. 2A. The spatial floating image display device 1000 shown in FIG. 4B is installed vertically so that the surface on which the spatial floating image 3 is formed faces the front (the direction of the user 230) of the spatial floating image display device 1000. That is, in FIG. 4B, for the spatial floating image display device, a transparent member 100 is installed on the front (the direction of the user 230) of the device. The spatial floating image 3 is formed on the user 230 side with respect to the surface of the transparent member 100 of the spatial floating image display device 1000. The light of the spatial floating image 3 travels in an obliquely upward direction. When the air operation detection sensor 1351 is provided as shown in the figure, the operation of the spatial floating image 3 by the finger of the user 230 can be detected. Here, as shown in FIG. 4B, the air operation detection sensor 1351 can use the reflection of the sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, since the nail has a higher reflectivity than the finger pad, the accuracy of touch detection can be improved by configuring it in this way.

[0103] Figure 4C is a diagram showing an example of the configuration of the spatial floating image display device. The spatial floating image display device 1000 shown in Figure 4C is equipped with an optical system corresponding to the optical system of Figure 2B. In the spatial floating image display device 1000 shown in Figure 4C, it is installed horizontally so that the surface on which the spatial floating image 3 is formed faces upward. That is, in Figure 4C, in the spatial floating image display device 1000, a transparent member 100 is installed on the upper surface of the device. The spatial floating image 3 is formed above the surface of the transparent member 100 of the spatial floating image display device 1000. The light of the spatial floating image 3 travels in an obliquely upward direction. When the air operation detection sensor 1351 is provided as shown in the figure, the operation of the spatial floating image 3 by the finger of the user 230 can be detected.

[0104] Figure 4D is a diagram showing an example of the configuration of the spatial floating image display device. The spatial floating image display device 1000 shown in Figure 4D is equipped with an optical system corresponding to the optical system of Figure 2B. The spatial floating image display device 1000 shown in Figure 4D is installed vertically so that the surface on which the spatial floating image 3 is formed faces the front of the spatial floating image display device 1000 (the direction of the user 230). That is, in Figure 4D, in the spatial floating image display device 1000, a transparent member 100 is installed on the front of the device (the direction of the user 230). The spatial floating image 3 is formed on the user 230 side with respect to the surface of the transparent member 100 of the spatial floating image display device 1000. The light of the spatial floating image 3 travels in an obliquely upward direction. When the air operation detection sensor 1351 is provided as shown in the figure, the operation of the spatial floating image 3 by the finger of the user 230 can be detected. Here, as shown in Figure 4D, the air operation detection sensor 1351 can utilize the reflection of the sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, since the nail has a higher reflectivity than the fingertip, the accuracy of touch detection can be improved by configuring it in this way.

[0105] FIG. 4E is a diagram showing an example of the configuration of the spatial floating image display device. The spatial floating image display device 1000 shown in FIG. 4E is equipped with an optical system corresponding to the optical system of FIG. 2C. In the spatial floating image display device 1000 shown in FIG. 4E, it is installed horizontally so that the surface on which the spatial floating image 3 is formed faces upward. That is, in FIG. 4E, in the spatial floating image display device 1000, the transparent member 100 is installed on the upper surface of the device. The spatial floating image 3 is formed above the surface of the transparent member 100 of the spatial floating image display device 1000. The light of the spatial floating image 3 travels in the directly upward direction. When the air operation detection sensor 1351 is provided as shown in the figure, the operation of the spatial floating image 3 by the finger of the user 230 can be detected.

[0106] FIG. 4F is a diagram showing an example of the configuration of the spatial floating image display device. The spatial floating image display device 1000 shown in FIG. 4F is equipped with an optical system corresponding to the optical system of FIG. 2C. The spatial floating image display device 1000 shown in FIG. 4F is installed vertically so that the surface on which the spatial floating image 3 is formed faces the front (the direction of the user 230) of the spatial floating image display device 1000. That is, in FIG. 4F, in the spatial floating image display device 1000, the transparent member 100 is installed on the front (the direction of the user 230) of the device. The spatial floating image 3 is formed on the user 230 side with respect to the surface of the transparent member 100 of the spatial floating image display device 1000. The light of the spatial floating image 3 travels in the direction in front of the user's hand. When the air operation detection sensor 1351 is provided as shown in the figure, the operation of the spatial floating image 3 by the finger of the user 230 can be detected.

[0107] FIG. 4G is a diagram showing an example of the configuration of the spatial floating image display device. The spatial floating image display device 1000 shown in FIG. 4G is equipped with an optical system corresponding to the optical system of FIG. 2C. In the optical systems of the spatial floating image display devices from FIGS. 4A to 4F, the central optical path of the image light emitted from the display device 1 was on the yz plane. That is, within the optical systems of the spatial floating image display devices from FIGS. 4A to 4F, the image light traveled in the front-rear direction and the up-down direction as seen by the user. In contrast, in the optical system of the spatial floating image display device shown in FIG. 4G, the central optical path of the image light emitted from the display device 1 is on the xy plane. That is, within the optical system of the spatial floating image display device shown in FIG. 4G, the image light travels in the left-right direction and the front-rear direction as seen by the user. In the spatial floating image display device 1000 shown in FIG. 4G, the surface on which the spatial floating image 3 is formed is installed so as to face the front of the device (the direction of the user 230). That is, in FIG. 4G, in the spatial floating image display device 1000, the transparent member 100 is installed on the front of the device (the direction of the user 230). The spatial floating image 3 is formed on the user side with respect to the surface of the transparent member 100 of the spatial floating image display device 1000. The light of the spatial floating image 3 travels in the direction in front of the user. When the air operation detection sensor 1351 is provided as shown in the figure, the operation of the spatial floating image 3 by the finger of the user 230 can be detected.

[0108] FIG. 4H is a diagram showing an example of the configuration of the floating image display device. The floating image display device 1000 in FIG. 4H is different from the floating image display device in FIG. 4G in that it has a window with a transparent plate 100B such as glass or plastic on the back of the device (the side opposite to the position where the user 230 views the floating image 3, that is, the side opposite to the traveling direction of the image light of the floating image 3 toward the user 230). Since the other configurations are the same as those of the floating image display device in FIG. 4G, repeated explanations are omitted. The floating image display device 1000 in FIG. 4H includes a window having a transparent plate 100B at a position opposite to the traveling direction of the image light of the floating image 3 with respect to the floating image 3. Therefore, when the user 230 views the floating image 3, the scenery behind the floating image display device 1000 can be recognized as the background of the floating image 3. Therefore, the user 230 can recognize that the floating image 3 is floating in the air in front of the scenery behind the floating image display device 1000. Thereby, the floating feeling of the floating image 3 in the air can be emphasized more.

[0109] Note that depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, a part of the image light output from the display device 1 may be reflected by the polarization separation member 101B and may go toward the transparent plate 100B. Depending on the coating performance of the surface of the transparent plate 100B, this light may be reflected again on the surface of the transparent plate 100B and may be visually recognized by the user 230 as stray light. Therefore, in order to prevent the stray light, the window on the back of the floating image display device 1000 may be configured not to be provided with the transparent plate 100B.

[0110] FIG. 4I is a diagram showing an example of the configuration of the floating image display device. The floating image display device 1000 in FIG. 4I is different from the floating image display device in FIG. 4H in that an opening / closing door 1410 for light shielding is provided in the window of the transparent plate 100B arranged on the back of the device (the side opposite to the position where the user 230 views the floating image 3). Since the other configurations are the same as those of the floating image display device in FIG. 4H, repeated explanations are omitted.

[0111] The opening / closing door 1410 of the spatial floating image display device 1000 in Fig. 4I has, for example, a light-shielding plate, and is provided with a mechanism for moving (sliding), rotating, or attaching / detaching the light-shielding plate, so that the window (rear-side window) of the transparent plate 100B located on the back side of the spatial floating image display device 1000 can be switched between an open state and a light-shielding state. The movement (sliding) or rotation of the light-shielding plate by the opening / closing door 1410 may be an electric type driven by a motor (not shown). The motor may be controlled by the control unit 1110 in Fig. 3. In the example of Fig. 4I, the number of light-shielding plates of the opening / closing door 1410 is disclosed as an example of two. In contrast, the number of light-shielding plates of the opening / closing door 1410 may be one.

[0112] For example, when the view seen through the window of the transparent plate 100B of the spatial floating image display device 1000 is outdoors, the brightness of sunlight varies depending on the weather. When the outdoor sunlight is strong, the background of the spatial floating image 3 may become too bright, and the visibility of the spatial floating image 3 for the user 230 may decrease. In such a case, if the rear-side window is shielded from light by moving (sliding), rotating, or attaching the light-shielding plate of the opening / closing door 1410, the background of the spatial floating image 3 will become darker, so that the visibility of the spatial floating image 3 can be relatively increased. Such a shielding operation by the light-shielding plate of the opening / closing door 1410 may be directly performed by the force of the user 230's hand. The control unit 1110 may control a motor (not shown) according to an operation input via the operation input unit 1107 in Fig. 3 to perform a shielding operation by the light-shielding plate of the opening / closing door 1410.

[0113] An illuminance sensor may be provided on the back side (opposite side of the user 230) of the spatial floating image display device 1000, such as near the rear-side window, to measure the brightness of the space in front of the rear-side window. In this case, the control unit 1110 in Fig. 3 may control a motor (not shown) according to the detection result of the illuminance sensor to perform an opening / closing operation by the light-shielding plate of the opening / closing door 1410. By controlling the opening / closing operation by the light-shielding plate of the opening / closing door 1410 in this way, it is possible to more suitably maintain the visibility of the spatial floating image 3 without the user 230 manually performing the opening / closing operation of the light-shielding plate of the opening / closing door 1410.

[0114] Alternatively, the light-shielding plate by the opening and closing door 1410 may be manually detachable. Depending on the usage purpose and installation environment of the spatial floating image display device 1000, the user can select whether to keep the rear window in an open state or a light-shielded state. If it is planned to use the rear window in a light-shielded state for a long time, the detachable light-shielding plate may be fixed in the light-shielded state. Also, if it is planned to use the rear window in an open state for a long time, it may be used with the detachable light-shielding plate removed. The attachment and detachment of the light-shielding plate may use screws, a hook structure, or a fitting structure.

[0115] In addition, in the example of the spatial floating image display device 1000 in FIG. 4I, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, a part of the image light output from the display device 1 may be reflected by the polarization separation member 101B and directed toward the transparent plate 100B. Depending on the coating performance of the surface of the transparent plate 100B, this light may be reflected again on the surface of the transparent plate 100B and visually recognized by the user 230 as stray light. Therefore, in order to prevent the stray light, the window on the back of the spatial floating image display device 1000 may be configured not to be provided with the transparent plate 100B. The above-described opening and closing door 1410 may be provided in the window without the transparent plate 100B. In order to prevent the stray light, it is desirable that the inner surface of the housing of the light-shielding plate of the above-described opening and closing door 1410 has a coating or material with a low light reflectance.

[0116] FIG. 4J is a diagram showing an example of the configuration of a spatial floating image display device. The spatial floating image display device 1000 in FIG. 4J is different in that an electronically controlled variable transmittance device 1620 is arranged in the rear window of the spatial floating image display device in FIG. 4H instead of arranging a transparent plate 100B made of glass or plastic. Since the other configurations are the same as those of the spatial floating image display device in FIG. 4H, repeated description is omitted. An example of the electronically controlled variable transmittance device 1620 is a liquid crystal shutter or the like. Although the electronically controlled variable transmittance device 1620 is not shown in FIG. 3, when this is provided, it may be configured to be connected to other processing units such as the control unit 1110 as a component of the spatial floating image display device 1000 in FIG. 3.

[0117] The liquid crystal shutter can control the light transmittance by controlling the voltage of the liquid crystal element sandwiched between two polarizing plates. Therefore, if the liquid crystal shutter is controlled to increase the transmittance, the background of the spatial floating image 3 will be in a state where the scenery through the rear window can be seen through. Also, if the liquid crystal shutter is controlled to decrease the transmittance, the scenery through the rear window can be made invisible as the background of the spatial floating image 3.

[0118] In addition, since the liquid crystal shutter can be controlled in intermediate tones, it can also be set to a state such as a transmittance of 50%. For example, the control unit 1110 may control the transmittance of the electronically controlled variable transmittance device 1620 according to the operation input via the operation input unit 1107 in FIG. 3. With this configuration, when the user wants to see the scenery through the rear window as the background of the spatial floating image 3, but the scenery through the rear window as the background is too bright and the visibility of the spatial floating image 3 is reduced, the visibility of the spatial floating image 3 can be adjusted by adjusting the transmittance of the electronically controlled variable transmittance device 1620.

[0119] An illuminance sensor may be provided on the rear side of the spatial floating image display device 1000 (the opposite side of the user 230), such as near the rear window, to measure the brightness of the space in front of the rear window. In this case, the control unit 1110 in FIG. 3 may control the transmittance of the electronically controlled variable transmittance device 1620 according to the detection result of the illuminance sensor. In this way, even if the user 230 does not perform an operation input via the operation input unit 1107 in FIG. 3, the transmittance of the electronically controlled variable transmittance device 1620 can be adjusted according to the brightness of the space in front of the rear window, so that the visibility of the spatial floating image 3 can be maintained more suitably.

[0120] Also, in the above example, an example of a liquid crystal shutter was described as the electronically controlled transmittance variable device 1620. On the other hand, as another example of the electronically controlled transmittance variable device 1620, electronic paper may be used. Even when using electronic paper, the same effects as described above can be obtained. Moreover, the power consumption for maintaining the intermediate state of electronic paper is very small. Therefore, a low-power spatial floating image display device can be realized compared to the case where a liquid crystal shutter is adopted.

[0121] FIG. 4K is a diagram showing an example of the configuration of a spatial floating image display device. The spatial floating image display device 1000 in FIG. 4K is different from the spatial floating image display device in FIG. 4G in that it has a transmissive self-emitting image display device 1650 instead of the transparent member 100. Since the other configurations are the same as those of the spatial floating image display device in FIG. 4G, repeated descriptions are omitted.

[0122] In the spatial floating image display device 1000 of FIG. 4K, after the image light beam passes through the display surface of the transmissive self-emitting image display device 1650, the spatial floating image 3 is formed outside the spatial floating image display device 1000. That is, when an image is being displayed on the transmissive self-emitting image display device 1650, which is a two-dimensional plane display, the spatial floating image 3 can be displayed as an image that pops out further on the user side of the image of the transmissive self-emitting image display device 1650. At this time, the user 230 can simultaneously view two images with different depths. The transmissive self-emitting image display device 1650 may be configured using existing technologies such as transmissive organic EL panels disclosed in, for example, Japanese Patent Application Laid-Open No. 2014-216761. When providing the transmissive self-emitting image display device 1650, it may be configured to be connected to other processing units such as the control unit 1110 as a component of the spatial floating image display device 1000 in FIG. 3.

[0123] Here, if an effect such as moving only objects such as characters to the front space floating image 3 after displaying both the background and objects such as characters on the transmissive self-luminous video display device 1650 is performed, an effective surprise effect video experience can be provided to the user 230.

[0124] Also, if the inside of the space floating video display device 1000 (housing 1190) is in a light-shielded state, the background of the transmissive self-luminous video display device 1650 will become sufficiently dark. Therefore, when no video is displayed on the display device 1 or the light source of the display device 1 is turned off and only the transmissive self-luminous video display device 1650 displays video, to the user 230, the transmissive self-luminous video display device 1650 appears to be a normal two-dimensional flat display rather than a transmissive display. Note that since the space floating video 3 in the embodiment of the present invention is displayed as an optical image of a real image in a space without a screen, if the light source of the display device 1 is turned off, the planned display position of the space floating video 3 will be an empty space. Therefore, when the transmissive self-luminous video display device 1650 is used to display video as if it were a general two-dimensional flat display, by suddenly displaying characters, objects, etc. as the space floating video 3 in the air, an effective surprise effect video experience can be provided to the user 230.

[0125] Note that the darker the interior of the spatial floating image display device 1000 is, the more the transmissive self-emitting image display device 1650 appears like a two-dimensional flat display. Therefore, an absorption-type polarizing plate (not shown) that transmits the polarization of the image light reflected by the polarization separation member 101B and absorbs a polarization having a phase difference of about 90° from the polarization may be provided on the inner side surface of the transmissive self-emitting image display device 1650 with respect to the spatial floating image display device 1000 (the incident surface of the image light reflected by the polarization separation member 101B to the transmissive self-emitting image display device 1650, that is, the surface of the transmissive self-emitting image display device 1650 opposite to the spatial floating image 3). In this way, the influence on the image light forming the spatial floating image 3 is not so great, but the light incident from the outside into the interior of the spatial floating image display device 1000 through the transmissive self-emitting image display device 1650 can be significantly reduced, the interior of the spatial floating image display device 1000 can be made darker, which is preferable.

[0126] FIG. 4L is a diagram showing an example of the configuration of a spatial floating image display device. The spatial floating image display device 1000 in FIG. 4L is a modified example of the spatial floating image display device in FIG. 4K. The orientation of the component arrangement in the spatial floating image display device 1000 is different from that of the spatial floating image display device in FIG. 4K and is similar to the arrangement of the spatial floating image display device in FIG. 4F. Since the functions, operations, etc. of each component are the same as those of the spatial floating image display device in FIG. 4K, repeated description will be omitted.

[0127] Also in the spatial floating image display device of FIG. 4L, after the image light beam passes through the transmissive self-emitting image display device 1650, the spatial floating image 3 is formed on the user 230 side of the transmissive self-emitting image display device 1650.

[0128] In the example of the spatial floating image display device of FIG. 4K or the example of the spatial floating image display device of FIG. 4L, from the user 230, the spatial floating image 3 is displayed overlapping in front of the image of the transmissive self-luminous image display device 1650. Here, the position of the spatial floating image 3 and the position of the image of the transmissive self-luminous image display device 1650 are configured to have a difference in the depth direction. Therefore, when the user 230 moves the head (the position of the viewpoint), the depth of the two images can be recognized due to the parallax. Therefore, by displaying two images with different depth positions, a three-dimensional visual experience can be more suitably provided to the user without the need for stereoscopic glasses or the like, with the naked eye.

[0129] FIG. 4M is a diagram showing an example of the configuration of the spatial floating image display device. The spatial floating image display device 1000 in FIG. 4M is provided with a second display device 1680 on the back side (for example, the back panel of the housing 1190) as viewed from the user 230 with respect to the polarization separation member 101B of the spatial floating image display device in FIG. 4G. Since the other configurations are the same as those of the spatial floating image display device in FIG. 4G, repeated descriptions are omitted.

[0130] In the configuration example shown in FIG. 4M, the second display device 1680 is provided on the back side of the display position of the spatial floating image 3, and the image display surface is directed toward the spatial floating image 3. With such a configuration, as viewed from the user 230, the image of the second display device 1680 and the spatial floating image 3 can be visually recognized by overlapping the images displayed at two different depth positions. That is, it can be said that the second display device 1680 is arranged in a direction to display an image in the direction of the user 230 who views the spatial floating image 3. When the second display device 1680 is provided, it may be configured to be connected to other processing units such as the control unit 1110 as a component of the spatial floating image display device 1000 in FIG. 3.

[0131] Note that the image light of the second display device 1680 of the spatial floating image display device 1000 in FIG. 4M is visually recognized by the user 230 after passing through the polarization separation member 101B. Therefore, in order for the image light of the second display device 1680 to more preferably pass through the polarization separation member 101B, it is desirable that the image light output from the second display device 1680 be polarized light of a polarization wave having a vibration direction in which the polarization separation member 101B more preferably passes through. That is, it is desirable that the polarization wave be polarized light having the same vibration direction as the polarization wave of the image light output from the display device 1. For example, when the image light output from the display device 1 is S-polarized light, it is desirable that the image light output from the second display device 1680 is also S-polarized light. Further, when the image light output from the display device 1 is P-polarized light, it is desirable that the image light output from the second display device 1680 is also P-polarized light.

[0132] The example of the spatial floating image display device in FIG. 4M also has the same effect as the example of the spatial floating image display device in FIG. 4K and the example of the spatial floating image display device in FIG. 4L in that a second image is displayed behind the spatial floating image 3. However, unlike the example of the spatial floating image display device in FIG. 4K and the example of the spatial floating image display device in FIG. 4L, in the example of the spatial floating image display device in FIG. 4M, the light beam of the image light for forming the spatial floating image 3 does not pass through the second display device 1680. Therefore, the second display device 1680 does not need to be a transmissive self-emitting image display device, and a liquid crystal display, which is a two-dimensional plane display, may be used. The second display device 1680 may also be an organic EL display. Therefore, in the example of the spatial floating image display device in FIG. 4M, it is possible to realize the spatial floating image display device 1000 at a lower cost than the example of the spatial floating image display device in FIG. 4K and the example of the spatial floating image display device in FIG. 4L.

[0133] Here, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, a part of the image light output from the display device 1 may be reflected by the polarization separation member 101B and may go toward the second display device 1680. This light (a part of the image light) may be reflected again on the surface of the second display device 1680 and may be visually recognized by the user as stray light.

[0134] Therefore, in order to prevent such stray light, an absorption-type polarizing plate may be provided on the surface of the second display device 1680. In this case, the absorption-type polarizing plate may be an absorption-type polarizing plate that transmits the polarization of the video light output from the second display device 1680 and absorbs the polarization whose phase is 90° different from the polarization of the video light output from the second display device 1680. When the second display device 1680 is a liquid crystal display, there is also an absorption-type polarizing plate on the video emission side inside the liquid crystal display. However, when there is a cover glass (cover glass on the video display surface side) on the further emission surface of the absorption-type polarizing plate on the video emission side inside the liquid crystal display, it is impossible to prevent the stray light generated by the reflection of the cover glass by the light from outside the liquid crystal display. Therefore, it is necessary to separately provide the above-described absorption-type polarizing plate on the surface of the cover glass.

[0135] In addition, when displaying a video on the second display device 1680 which is a two-dimensional plane display, the spatial floating video 3 can be displayed as a video further on the user side of the video of the second display device 1680. At this time, the user 230 can simultaneously view two videos with different depth positions. By displaying a character on the spatial floating video 3 and displaying a background on the second display device 1680, an effect can be provided such that the user 230 visually recognizes the space where the character exists as if it were three-dimensional.

[0136] Also, after displaying both an object such as a background and a character on the second display device 1680, an effect such as displaying only the object such as a character moving to the spatial floating video 3 on the front side can be performed, thereby providing the user 230 with an effective video experience with a surprise effect.

[0137] Next, FIG. 4N is a diagram showing an example of the configuration of a spatial floating image display device. The spatial floating image display device 1000 in FIG. 4N is a spatial floating image display device that employs the optical system of FIG. 2D. Similar to the example of the spatial floating image display device that employs the optical systems of FIGS. 2A to 2C, the image light that has passed through the transparent member 100 forms an image in the air as the spatial floating image 3. Also, using the sensing light of the air operation detection sensor 1351 disposed on the back side of the transparent member 100 as viewed from the user, the operation of the spatial floating image 3 by the finger 9004 of the user can be detected.

[0138] In the example of the spatial floating image display device that employs the optical systems of FIGS. 2A to 2C and also in the example of the spatial floating image display device that employs the optical system of FIG. 2D, the spatial floating image 3 forms an image in front of the transparent member 100, and using the sensing light of the air operation detection sensor 1351 disposed on the back side of the transparent member 100 as viewed from the user, the operation of the spatial floating image 3 by the finger of the user can be detected.

[0139] The spatial floating image display device that employs the optical system of FIG. 2D has a different optical system from the spatial floating image display device in which the optical systems of FIGS. 2A to 2C are disposed on the back side of the transparent member 100 as viewed from the user. However, the usability of the spatial floating image display device that employs the optical system of FIG. 2D as viewed from the user will be substantially the same as that of the spatial floating image display device that employs the optical systems of FIGS. 2A to 2C.

[0140] Next, FIG. 4O is a diagram showing an example of the configuration of a spatial floating image display device. FIG. 4O is a diagram showing the internal optical system configuration of the spatial floating image display device 1000 in FIG. 4N. The spatial floating image display device 1000 shown in FIG. 4O is equipped with an optical system corresponding to the optical system of FIG. 2D. In the spatial floating image display device 1000 shown in FIG. 4O, it is installed horizontally so that the surface on which the spatial floating image 3 is formed faces upward.

[0141] That is, in FIG. 4O, the transparent member 100 of the spatial floating image display device 1000 is installed on the upper surface of the device. The spatial floating image 3 is formed above the surface of the transparent member 100 of the spatial floating image display device 1000. The light of the spatial floating image 3 travels in an obliquely upward direction. When the air operation detection sensor 1351 is provided as shown in the figure, the operation of the spatial floating image 3 by the finger of the user 230 can be detected.

[0142] Here, the configuration of FIG. 4O is compared with the configuration of FIG. 4A to confirm the differences. In FIG. 4A, the display device 1 and the spatial floating image 3 are in a plane-symmetrical relationship with respect to the surface of the polarization separation member 101. In contrast, in FIG. 4O, the display device 1 and the spatial floating image 3 are in a plane-symmetrical relationship with respect to the surface of the retroreflective plate 5. Also, in the configuration of FIG. 4A, there are a retroreflective plate 2 and a λ / 4 plate 21, but these do not exist in FIG. 4O. Further, in FIG. 4A, it is more preferable to have an absorption type polarizing plate 12, but in FIG. 4O, the absorption type polarizing plate 12 is not particularly necessary.

[0143] To replace the optical system of FIG. 2A in the configuration of FIG. 4A with the optical system of FIG. 2D and replace it with the configuration of FIG. 4O, the following may be done. That is, the polarization separation member 101 in the configuration of FIG. 4A may be replaced with a retroreflective plate 5, and the retroreflective plate 2 and the λ / 4 plate 21 may be removed from the configuration of FIG. 4A. The absorption type polarizing plate 12 may or may not be present. By performing the replacement based on this idea, the optical systems of FIGS. 2A to 2C mounted in the configurations of the spatial floating image display devices of FIGS. 4A to 4G can be replaced with the optical system of FIG. 2D, and can be replaced with a spatial floating image display device equipped with the optical system of FIG. 2D. At this time, in FIGS. 4A and 4B, the polarization separation member 101 may be replaced with a retroreflective plate 5, and in FIGS. 4C to 4G, the polarization separation member 101B may be replaced with a retroreflective plate 5.

[0144] For example, FIG. 4P is a diagram showing an example of the configuration of a spatial floating image display device. The spatial floating image display device 1000 shown in FIG. 4P is equipped with an optical system corresponding to the optical system of FIG. 2D. FIG. 4P is obtained by replacing the optical system of FIG. 2A mounted on the configuration of the spatial floating image display device of FIG. 4B with the optical system of FIG. 2D. The spatial floating image display device 1000 shown in FIG. 4P is installed vertically so that the surface on which the spatial floating image 3 is formed faces the front (the direction of the user 230) of the spatial floating image display device 1000. That is, in FIG. 4P, the transparent member 100 of the spatial floating image display device is installed on the front (the direction of the user 230) of the device. The spatial floating image 3 is formed on the surface of the transparent member 100 of the spatial floating image display device 1000 on the user 230 side. The light of the spatial floating image 3 travels in an obliquely upward direction. When the air operation detection sensor 1351 is provided as shown in the figure, the operation of the spatial floating image 3 by the finger of the user 230 can be detected. Here, as shown in FIG. 4P, the air operation detection sensor 1351 can utilize the reflection of the sensing light by the user's nail for touch detection by sensing the finger of the user 230 from above. Generally, since the nail has a higher reflectivity than the pulp of the finger, the accuracy of touch detection can be improved by configuring it in this way.

[0145] According to the configurations of the spatial floating image display devices of FIGS. 4N to 4P, a user-friendly spatial floating image display device using the optical system of FIG. 2D can be realized.

[0146] <Display device> Next, the display device 1 of this embodiment will be described with reference to the drawings. The display device 1 of this embodiment includes a liquid crystal display panel 11, which is an image display element 11, and a light source device 13 that constitutes the light source of the liquid crystal display panel 11. In FIG. 5, the light source device 13 is shown as an exploded perspective view together with the liquid crystal display panel 11.

[0147] The image display element 11, which is this liquid crystal display panel 11, receives, as shown by arrow 30 in FIG. 5, an illumination light beam from a light source device 13 that is a backlight device and has diffusion characteristics with a sandwiching angle, that is, characteristics similar to a laser beam with strong directivity (in other words, rectilinearity) and with the polarization planes aligned in one direction. The image display element 11, which is this liquid crystal display panel 11, modulates the received illumination light beam according to the input video signal. The modulated video light is reflected by the retroreflective plate 2, passes through the transparent member 100, and forms a spatial floating image that is a real image (see FIG. 1).

[0148] Also, in FIG. 5, the display device 1 includes the light source device 13 and the liquid crystal display panel 11, and further includes a light direction conversion panel 54 that controls the directivity characteristics of the emitted light beam from the light source device 13, and an optional sandwiching angle diffuser (not shown). That is, polarizing plates are provided on both sides of the liquid crystal display panel 11, and as shown by arrow 30 in FIG. 5, the video light of a specific polarization modulates the light intensity according to the video signal and is emitted. As a result, a desired video is projected as light of a specific polarization with high directivity (rectilinearity) through the light direction conversion panel 54 toward the retroreflective plate 2. After being reflected by the retroreflective plate 2, it passes through and forms a spatial floating video 3 toward the eyes of an observer outside the store (space) in FIG. 1. A protective cover 50 (see FIGS. 6 and 7) may be provided on the surface of the above-described light direction conversion panel 54.

[0149] <Example 1 of display device> FIG. 6 shows an example of the specific configuration of the display device 1. In FIG. 6, a liquid crystal display panel 11 and a light direction conversion panel 54 are arranged on the light source device 13 of FIG. 5. This light source device 13 is configured by housing, for example, an LED element 201 and a light guide 203 inside a plastic case or the like. On the end face of the light guide 203, as shown in FIG. 5 and the like, in order to convert the divergent light from each LED element 201 into a substantially parallel light beam, it has a shape in which the cross-sectional area gradually increases toward the light receiving part, and has a lens shape that has an effect of gradually reducing the divergence angle by total internal reflection a plurality of times when propagating inside. A liquid crystal display panel 11 constituting such a display device 1 is attached to the upper surface of the display device 1. Further, on one side surface (the left end surface in this example) of the light source device 13, an LED element 201 which is a semiconductor light source and an LED substrate 202 on which its control circuit is mounted are attached. A heat sink, which is a member for cooling the heat generated by the LED element 201 and the control circuit, may be attached to the outer surface of the LED substrate 202.

[0150] Further, on a frame (not shown) of the liquid crystal display panel 11 attached to the upper surface of the case of the light source device 13, the liquid crystal display panel 11 attached to the frame, and further, an FPC (Flexible Printed Circuits) (not shown) electrically connected to the liquid crystal display panel 11 and the like are attached and configured. That is, the liquid crystal display panel 11, which is the video display element 11, together with the LED element 201, which is a solid light source, generates a display video by modulating the intensity of transmitted light based on a control signal from a control circuit (video control unit 1160 in FIG. 3) that constitutes an electronic device. At this time, since the generated video light has a narrow diffusion angle and only a specific polarization component, a novel video display device close to a surface-emitting laser video source driven by a video signal can be obtained. At present, it is technically and safety-wise impossible to obtain a laser light beam of the same size as the image obtained by the above-described display device 1 using a laser device. Therefore, in this embodiment, for example, light close to the above-described surface-emitting laser video light is obtained from a light beam from a general light source equipped with an LED element.

[0151] Next, the configuration of the optical system housed in the case of the light source device 13 will be described in detail with reference to FIG. 7 together with FIG. 6. Since FIGS. 6 and 7 are cross-sectional views, only one of the plurality of LED elements 201 constituting the light source is shown, and these are converted into substantially parallel light (collimated light) by the shape of the light-receiving end face 203a of the light guide 203. Therefore, the light-receiving portion of the light guide end face and the LED element 201 are attached while maintaining a predetermined positional relationship.

[0152] Each of these light guides 203 is formed of a light-transmissive resin such as acrylic, for example. And the LED light-receiving surface at the end of this light guide 203 has, for example, an outer peripheral surface of a conical convex shape obtained by rotating a parabolic cross-section. At the top, it has a concave portion formed with a convex portion (that is, a convex lens surface) at the central portion thereof, and at the central portion of the flat portion, it has a convex lens surface protruding outward (or it may be a concave lens surface recessed inward) (not shown). The outer shape of the light-receiving portion of the light guide to which the LED element 201 is attached has a parabolic surface shape forming a conical outer peripheral surface, and is set within a range of angles that can totally reflect the light emitted from the LED element in the peripheral direction inside, or a reflecting surface is formed.

[0153] On the other hand, the LED elements 201 are respectively arranged at predetermined positions on the surface of the LED substrate 202, which is the circuit board thereof. This LED substrate 202 is arranged and fixed with respect to the light-receiving end face 203a, which is an LED collimator, such that the LED elements 201 on its surface are respectively located at the central portions of the aforementioned concave portions.

[0154] According to such a configuration, due to the shape of the light-receiving end face 203a of the light guide 203, the light emitted from the LED element 201 can be taken out as substantially parallel light, and the utilization efficiency of the generated light can be improved.

[0155] As described above, the light source device 13 is configured by attaching a light source unit in which a plurality of LED elements 201 as light sources are arranged on a light receiving end surface 203a, which is a light receiving portion provided on an end surface of the light guide 203. The divergent light beam from the LED element 201 is made into substantially parallel light by the lens shape of the light receiving end surface 203a of the light guide 203, and is guided inside the light guide 203 as indicated by the arrow. Then, the light beam is emitted toward the liquid crystal display panel 11 arranged substantially parallel to the light guide 203 by the light beam direction conversion means 204. By optimizing the distribution (in other words, density) of the light beam direction conversion means 204 according to the shape of the inside or surface of the light guide 203, the uniformity of the light beam incident on the liquid crystal display panel 11 can be controlled.

[0156] The above-described light beam direction conversion means 204 emits the light beam propagating inside the light guide 203 toward the liquid crystal display panel 11 arranged substantially parallel to the light guide 203 by the shape of the surface of the light guide 203 or by providing, for example, portions with different refractive indices inside the light guide 203. At this time, if the relative luminance ratio when comparing the luminance of the center of the screen and the peripheral portion of the screen in a state where the liquid crystal display panel 11 faces the center of the screen and the viewpoint is placed at a position equal to the diagonal dimension of the screen is 20% or more, there is no practical problem, and if it exceeds 30%, the characteristics are even more excellent.

[0157] Note that FIG. 6 is a cross-sectional layout view for explaining the configuration and operation of the light source of the present embodiment that performs polarization conversion in the light source device 13 including the above-described light guide 203 and LED element 201. In FIG. 6, the light source device 13 includes, for example, a light guide 203 provided with light beam direction conversion means 204 on its surface or inside formed of plastic or the like, an LED element 201 as a light source, a reflection sheet 205, a retardation plate 206, a lenticular lens, and the like. A liquid crystal display panel 11 having polarizing plates on the light source light incident surface and the video light emitting surface is attached to the upper surface of the light source device 13.

[0158] Also, a film or sheet-like reflective polarizing plate 49 is provided on the light source light incident surface (the lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, and selectively reflects one-sided polarized wave (for example, P wave) 212 of the natural light beam 210 emitted from the LED element 201. The reflected light is reflected again by a reflection sheet 205 provided on one (the lower side in the figure) surface of the light guide 203 so as to be directed toward the liquid crystal display panel 11. Therefore, a retardation plate (λ / 4 plate) is provided between the reflection sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49. By reflecting the reflected light (reflected light beam) with the reflection sheet 205 and passing it through the retardation plate (λ / 4 plate) twice, the P polarization is converted to S polarization. Thereby, the utilization efficiency of the light source light as video light is improved. The video light beam whose light intensity is modulated by the video signal in the liquid crystal display panel 11 is emitted as shown by the arrow 213 in FIG. 6 and enters the retroreflective plate 2. After being reflected by the retroreflective plate 2, a spatial floating image that is a real image can be obtained.

[0159] FIG. 7 is a cross-sectional layout diagram for explaining the configuration and operation of the light source of the present embodiment that performs polarization conversion in the light source device 13 including the light guide 203 and the LED element 201, similar to FIG. 6. Similarly, the light source device 13 is composed of, for example, a light guide 203 provided with a light beam direction conversion means 204 on its surface or inside formed of plastic or the like, an LED element 201 as a light source, a reflection sheet 205, a retardation plate 206, a lenticular lens, and the like. A liquid crystal display panel 11 having polarizing plates on the light source light incident surface and the video light emission surface is attached to the upper surface of the light source device 13.

[0160] Also, a film or sheet-like reflective polarizing plate 49 is provided on the light source light incident surface (the lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, and selectively reflects one-sided polarized wave (for example, S wave) 211 among the natural light beams 210 emitted from the LED element 201. That is, in the example of FIG. 7, the selective reflection characteristics of the reflective polarizing plate 49 are different from those in FIG. 7. The reflected light is reflected by a reflection sheet 205 provided on one surface (the lower side in the figure) of the light guide 203 and then heads toward the liquid crystal display panel 11 again. A retardation plate (λ / 4 plate) is provided between the reflection sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizing plate 49. The reflected light (reflected light beam) is reflected by the reflection sheet 205 and passed through the retardation plate (λ / 4 plate) twice to convert from S polarization to P polarization. Thereby, the utilization efficiency of the light source light as video light is improved. The video light beam intensity-modulated by the video signal in the liquid crystal display panel 11 is emitted as shown by the arrow 214 in FIG. 7 and enters the retroreflective plate 2. After being reflected by the retroreflective plate 2, a spatial floating image that is a real image can be obtained.

[0161] In the light source device 13 shown in FIGS. 6 and 7, in addition to the action of the polarizing plate provided on the light incident surface of the corresponding liquid crystal display panel 11, the reflective polarizing plate reflects one-sided polarization components. Therefore, the theoretically obtainable contrast ratio is the product of the reciprocal of the cross transmittance of the reflective polarizing plate and the reciprocal of the cross transmittance obtained by the two polarizing plates attached to the liquid crystal display panel 11. Thereby, high contrast performance can be obtained. Actually, it was experimentally confirmed that the contrast performance of the displayed image was improved by 10 times or more. As a result, a high-quality video comparable to that of the self-luminous organic EL was obtained.

[0162] <Example 2 of the display device> FIG. 8 shows another example of the specific configuration of the display device 1. The light source device 13 of this display device 1 is configured by housing an LED, a collimator, a composite diffuser block, a light guide, etc. in a case made of, for example, plastic. A liquid crystal display panel 11 is attached to the upper surface of the light source device 13. Further, on one side surface of the case of the light source device 13, an LED element 201 which is a semiconductor light source and an LED substrate 202 on which a control circuit of the LED element 201 is mounted are attached, and on the outer surface of the LED substrate 202, a heat sink 103 which is a member for cooling the heat generated by the LED element 201 and the control circuit is attached.

[0163] In addition, on the liquid crystal display panel frame attached to the upper surface of the case of the light source device 13, the liquid crystal display panel 11 attached to the frame and further an FPC 403 electrically connected to the liquid crystal display panel 11 are attached and configured. That is, the liquid crystal display panel 11 which is the video display element 11, together with the LED element 201 which is a solid light source, modulates the intensity of transmitted light based on a control signal from a control circuit (not shown) that constitutes the electronic device, thereby generating a display video.

[0164] <Example 3 of Display Device> Subsequently, another example (Example 3 of the display device) of the specific configuration of the display device 1 will be described with reference to FIG. 9. The light source device of this display device 1 converts the divergent light beam (mixed P-polarized light and S-polarized light) from the LED 201 into a substantially parallel light beam by a collimator (LED collimator) 18 and reflects it toward the liquid crystal display panel 11 by the reflecting surface of the reflective light guide 304. The reflected light enters a reflective polarizing plate 49 disposed between the liquid crystal display panel 11 and the reflective light guide 304. The reflective polarizing plate 49 transmits light of a specific polarization (for example, P-polarized light) and makes the transmitted polarized light enter the liquid crystal display panel 11. Here, light of other polarizations (for example, S-polarized light) other than the specific polarization is reflected by the reflective polarizing plate 49 and heads back to the reflective light guide 304 again.

[0165] The reflective polarizing plate 49 is installed at an inclination with respect to the liquid crystal display panel 11 so as not to be perpendicular to the principal ray of light from the reflecting surface of the reflective light guide 304. Then, the principal ray of the light reflected by the reflective polarizing plate 49 enters the transmissive surface of the reflective light guide 304. The light that has entered the transmissive surface of the reflective light guide 304 passes through the back surface of the reflective light guide 304, passes through the λ / 4 plate 270 which is a retardation plate, and is reflected by the reflector 271. The light reflected by the reflector 271 passes through the λ / 4 plate 270 again and passes through the transmissive surface of the reflective light guide 304. The light that has passed through the transmissive surface of the reflective light guide 304 enters the reflective polarizing plate 49 again.

[0166] At this time, since the light that enters the reflective polarizing plate 49 again has passed through the λ / 4 plate 270 twice, the polarization is converted into a polarization (for example, P-polarization) that passes through the reflective polarizing plate 49. Therefore, the light whose polarization has been converted passes through the reflective polarizing plate 49 and enters the liquid crystal display panel 11. Regarding the polarization design related to polarization conversion, the polarization may be configured in the reverse manner (reversing S-polarization and P-polarization) from the above description.

[0167] As a result, the light from the LED 201 is aligned to a specific polarization (for example, P-polarization), enters the liquid crystal display panel 11, is luminance-modulated in accordance with the video signal, and a video is displayed on the panel surface. Similar to the above example, it has a plurality of LEDs 201 that constitute the light source, and these are attached at predetermined positions with respect to the corresponding collimators 18 of the plurality of collimators 18. However, in FIG. 9, only one LED 201 and one collimator 18 are shown because it is a longitudinal section.

[0168] Note that each of the collimators 18 is formed of, for example, a light-transmitting resin such as acrylic or glass. The collimator 18 may have an outer peripheral surface with a conical convex shape obtained by rotating a parabolic cross-section. Further, a concave portion having a convex portion (i.e., a convex lens surface) may be formed at the central portion of the top of the collimator 18 (the side facing the LED substrate 202). Further, a convex lens surface protruding outward (or a concave lens surface recessed inward may also be acceptable) is provided at the central portion of the flat surface portion of the collimator 18 (the side opposite to the above top). Note that the parabolic surface forming the conical outer peripheral surface of the collimator 18 is set within an angular range capable of total reflection of the light emitted from the LED 201 in the peripheral direction inside the collimator 18, or a reflective surface is formed.

[0169] Note that the LEDs 201 are respectively arranged at predetermined positions on the surface of the LED substrate 202, which is their circuit board. The LED substrate 202 is arranged and fixed with respect to the collimator 18 such that the LEDs 201 on its surface are respectively located at the central portion of the top of the conical convex shape (the concave portion if there is a concave portion at the top).

[0170] According to such a configuration, among the light emitted from the LED 201, particularly the light emitted from the central portion thereof, is condensed by the convex lens surface forming the outer shape of the collimator 18 to become parallel light. Further, the light emitted from other portions in the peripheral direction is reflected by the parabolic surface forming the conical outer peripheral surface of the collimator 18 and similarly condensed to become parallel light. In other words, according to the collimator 18 having a convex lens formed at its central portion and a parabolic surface formed at its peripheral portion, almost all of the light generated by the LED 201 can be taken out as parallel light, and the utilization efficiency of the generated light can be improved.

[0171] Furthermore, the light converted into substantially parallel light by the collimator 18 shown in FIG. 9 is reflected by the reflective light guide 304. Among this light, the light of a specific polarization state passes through the reflective polarizing plate 49 due to the action of the reflective polarizing plate 49, and the light of the other polarization state reflected by the action of the reflective polarizing plate 49 passes through the light guide 304 again. This light is reflected by a reflector 271 located at a position opposite to the liquid crystal display panel 11 with respect to the reflective light guide 304. At this time, the light is polarization-converted by passing through the λ / 4 plate 270, which is a retardation plate, twice. The light reflected by the reflector 271 passes through the light guide 304 again and enters the reflective polarizing plate 49 provided on the opposite surface. Since the incident light has been polarization-converted, it passes through the reflective polarizing plate 49 and is incident on the liquid crystal display panel 11 with its polarization directions aligned. As a result, all the light from the light source can be utilized, so the geometric optical utilization efficiency of the light is doubled. Also, since the polarization degree (extinction ratio) of the reflective polarizing plate 49 is also factored into the extinction ratio of the entire system, the contrast ratio of the entire display device is significantly improved by using the light source device of this embodiment. Note that by adjusting the surface roughness of the reflection surface of the reflective light guide 304 and the surface roughness of the reflector 271, the light reflection diffusion angle at each reflection surface can be adjusted. The surface roughness of the reflection surface of the reflective light guide 304 and the surface roughness of the reflector 271 may be adjusted for each design so that the uniformity of the light incident on the liquid crystal display panel 11 becomes more suitable.

[0172] Note that the λ / 4 plate 270, which is the retardation plate in FIG. 9, does not necessarily need to have a retardation of λ / 4 with respect to the polarization incident perpendicularly to the λ / 4 plate 270. In the configuration of FIG. 9, any retardation plate that changes the phase by 90° (λ / 2) when the polarization passes through it twice is acceptable. The thickness of the retardation plate may be adjusted according to the incident angle distribution of the polarization.

[0173] <Example 4 of the display device> Furthermore, another example (Example 4 of the display device) of the configuration of the optical system such as the light source device of the display device 1 will be described with reference to FIG. 10. Example 4 of the display device is a configuration example in the case where a diffusion sheet is used instead of the reflective light guide 304 in the light source device of Example 3 of the display device. Specifically, two optical sheets (in other words, diffusion sheets) that convert the diffusion characteristics in the vertical and horizontal directions (not shown in the front-back direction of the drawing) of the drawing are used on the light-emitting side of the light from the collimator 18. The two optical sheets are shown as an optical sheet 207A and an optical sheet 207B. The light from the collimator 18 is made to enter between the two optical sheets.

[0174] Note that the above optical sheet may be a single sheet instead of two sheets. In the case of a single-sheet configuration, the vertical and horizontal diffusion characteristics are adjusted by the fine shapes of the front and back surfaces of the single optical sheet. Also, multiple diffusion sheets may be used to share the function. Here, in the example of FIG. 10, regarding the reflection and diffusion characteristics due to the front and back surface shapes of the optical sheet 207A and the optical sheet 207B, it is advisable to optimize the design with the number of LEDs 201, the divergence angle from the LED substrate 202, and the optical specifications of the collimator 18 as design parameters so that the surface density of the light beam emitted from the liquid crystal display panel 11 becomes uniform. That is, in the example of FIG. 10, the diffusion characteristics are adjusted by the surface shapes of a plurality of diffusion sheets instead of the light guide.

[0175] In the example of FIG. 10, the polarization conversion is performed in the same manner as in Example 3 of the display device described above. That is, in the example of FIG. 10, the reflective polarizing plate 49 may be configured to have the characteristic of reflecting S-polarized light (transmitting P-polarized light). In that case, among the light emitted from the LED 201 which is the light source, the P-polarized light is transmitted, and the transmitted light enters the liquid crystal display panel 11. Among the light emitted from the LED 201 which is the light source, the S-polarized light is reflected, and the reflected light passes through the retardation plate 270 shown in FIG. 10. The light that has passed through the retardation plate 270 is reflected by the reflector 271. The light reflected by the reflector 271 is converted to P-polarized light by passing through the retardation plate 270 again. The polarization-converted light passes through the reflective polarizing plate 49 and enters the liquid crystal display panel 11.

[0176] Note that the quarter-wave plate 270, which is the retardation plate in Fig. 10, does not necessarily need to have a retardation of λ / 4 with respect to the polarized light incident perpendicularly to the quarter-wave plate 270. In the configuration of Fig. 10, any retardation plate that changes the phase by 90° (λ / 2) when the polarized light passes through it twice is acceptable. The thickness of the retardation plate may be adjusted according to the incident angle distribution of the polarized light. Also, in Fig. 10, regarding the polarization design related to polarization conversion, the polarization states (reversing S-polarization and P-polarization) may be configured in the reverse manner from the above description.

[0177] The light emitted from the liquid crystal display panel 11 has similar diffusion characteristics in both the horizontal direction of the screen (displayed on the X-axis in Fig. 12(a)) and the vertical direction of the screen (displayed on the Y-axis in Fig. 12(b)) in a general TV application device. In contrast, for the diffusion characteristics of the light beam emitted from the liquid crystal display panel 11 of this embodiment, for example, as shown in Example 1 of Fig. 12, by setting the viewing angle at which the luminance becomes 50% of the front view (angle 0 degrees) to 13 degrees, it becomes 1 / 5 compared to 62 degrees of a general TV application device. Similarly, for the viewing angle in the vertical direction, the reflection angle of the reflective light guide and the area of the reflection surface, etc. are optimized so that the upper viewing angle is suppressed to about 1 / 3 of the lower viewing angle with upper and lower non-uniformity. As a result, compared with a conventional liquid crystal TV, the amount of video light directed toward the monitoring direction is significantly improved, and the luminance becomes 50 times or more.

[0178] Furthermore, for the viewing angle characteristics shown in Example 2 of Fig. 12, by setting the viewing angle at which the luminance becomes 50% of the front view (angle 0 degrees) to 5 degrees, it becomes 1 / 12 compared to 62 degrees of a general TV application device. Similarly, for the viewing angle in the vertical direction, the reflection angle of the reflective light guide and the area of the reflection surface, etc. are optimized so that the viewing angle is suppressed to about 1 / 12 of a general TV application device with upper and lower uniformity. As a result, compared with a conventional liquid crystal TV, the amount of video light directed toward the monitoring direction is significantly improved, and the luminance becomes 100 times or more.

[0179] As described above, by using the viewing angle as the included angle, the amount of light flux directed toward the monitoring direction can be concentrated, so the light utilization efficiency is greatly improved. As a result, even when using a liquid crystal display panel for general TV applications, by controlling the light diffusion characteristics of the light source device, it is possible to achieve a significant increase in brightness with the same power consumption, and it can be made into a video display device corresponding to an information display system for bright outdoors.

[0180] When using a large liquid crystal display panel, the light around the screen is directed inward so that it heads toward the viewer when the viewer faces the center of the screen, thereby improving the overall uniformity of the screen brightness. FIG. 11 shows the convergence angles of the long side and the short side of the panel when the distance L from the panel to the viewer and the panel size (screen ratio 16:10) are used as parameters. When monitoring with the screen in a portrait orientation, the convergence angle can be set according to the short side. For example, when using a 22-inch panel in portrait orientation and the monitoring distance is 0.8 m, if the convergence angle is set to 10 degrees, the video light from the four corners of the screen can be effectively directed toward the viewer.

[0181] Similarly, when monitoring with a 15-inch panel in portrait orientation and the monitoring distance is 0.8 m, if the convergence angle is set to 7 degrees, the video light from the four corners of the screen can be effectively directed toward the viewer. As described above, by directing the video light around the screen to the viewer who is in the optimal position to monitor the center of the screen according to the size of the liquid crystal display panel and whether it is used in portrait or landscape orientation, the overall uniformity of the screen brightness can be improved.

[0182] As a basic configuration, as shown in FIG. 9, a light flux with an included angle directivity characteristic is incident on the liquid crystal display panel 11 by a light source device, and the video information displayed on the screen of the liquid crystal display panel 11 is modulated in brightness according to the video signal, and the spatially floating video obtained by reflecting with a retroreflective plate is displayed outdoors or indoors through a transparent member 100.

[0183] By using the display device and the light source device according to an embodiment of the present invention described above, it is possible to realize a spatially floating video display device with higher light utilization efficiency.

[0184] <Example of video display processing in a spatial floating video display device> Next, an example of a problem solved by the image processing of this embodiment will be described with reference to FIG. 13A. In the spatial floating video display device 1000 (FIGS. 3, 4A to 4P), when the back side of the spatial floating video 3 as viewed by the user is inside the housing of the spatial floating video display device 1000 and is sufficiently dark, the user visually recognizes that the background of the spatial floating video 3 is black.

[0185] Here, an example of displaying the character "Panda" 1525 in the spatial floating video 3 will be described with reference to FIG. 13A. First, the video control unit 1160 in FIG. 3 recognizes, as shown in FIG. 13A(1), the pixel region for drawing the image of the character "Panda" 1525 and the transparent information region 1520 which is the background image, by distinguishing the pixel region for drawing the image of the character "Panda" 1525 from the transparent information region 1520 which is the background image.

[0186] A method of distinguishing and recognizing the character image and the background image is, for example, configured in the image processing of the video control unit 1160 such that the background image layer and the layer of the character image in front of the background image layer can be processed as separate layers, and the character image and the background image may be distinguished and recognized based on the superimposition relationship when these layers are combined.

[0187] Here, the video control unit 1160 recognizes the pixels for drawing an object such as a character image as information different from the black and transparent information pixels. However, it is assumed that the luminance of both the black and transparent information pixels of the pixels for drawing the object is 0. In this case, when displaying the spatial floating video 3, there is no difference in luminance between the pixels for drawing black in the image of the character "panda" 1525 and the pixels in the transparent information area 1520 which is the background image. Therefore, in the spatial floating video 3, as shown in Fig. 13A(2), there is no luminance in either the pixels for drawing black in the image of the character "panda" 1525 or the pixels in the transparent information area 1520, and it is visually recognized by the user as an optically identical black space. That is, the part for drawing black in the image of the object, the character "panda" 1525, melts into the background, and only the part of the character "panda" 1525 that is not black is recognized as a video floating in the display area of the spatial floating video 3.

[0188] An example of the image processing in this embodiment will be described with reference to Fig. 13B. Fig. 13B is a diagram for explaining an example of image processing that more preferably resolves the problem that the black image area of the object melts into the background, which was described with reference to Fig. 13A. In Figs. 13B(1) and (2), the display state of the spatial floating video 3 is shown on the upper side, and the input / output characteristics of the image processing of the object image are shown on the lower side, respectively. Note that the image of the object (character "panda" 1525) and the corresponding data may be read from the storage unit 1170 or the memory 1109 in Fig. 3. Alternatively, it may be input from the video signal input unit 1131. Alternatively, it may be acquired via the communication unit 1132.

[0189] Here, in the state of Fig. 13B(1), the input / output characteristics of the image processing of the object image are in a linear state without any particular adjustment. In this case, it is the same display state as Fig. 13A(2), and the black image area of the object melts into the background. On the other hand, in Fig. 13B(2), the video control unit 1160 in this embodiment adjusts the input / output characteristics of the image processing for the image of the object (character "panda" 1525) to be as shown in the input / output characteristics in the lower part.

[0190] That is, the video control unit 1160 performs image processing of input / output characteristics, which has the characteristic of converting the pixels of the input image into output pixels with increased luminance values for the pixels in the low-luminance area of the image of the object (the character "panda" 1525). After the image of the object (the character "panda" 1525) has been subjected to the image processing of the input / output characteristics, the video including the image of the object (the character "panda" 1525) is input to the display device 1 and displayed. Then, as shown in the upper part of Fig. 13B(2), the display state of the floating-in-air video 3 is such that the luminance of the pixel area for drawing black in the image of the character "panda" 1525 increases. As a result, among the areas for drawing the image of the character "panda" 1525, the areas for drawing black can also be distinguished from the black of the background without melting into it and recognized by the user, making it possible to more suitably display the object.

[0191] That is, by using the image processing of Fig. 13B(2), the area for displaying the image of the object, the character "panda" 1525, can be recognized as distinct from the black background inside the housing of the floating-in-air video display device 1000 through the window, improving the visibility of the object. Therefore, for example, before the image processing (i.e., when the image of the object or the corresponding data is read from the storage unit 1170 or the memory 1109 in Fig. 3, or when the image of the object is input from the video signal input unit 1131, or when the data of the object is acquired via the communication unit 1132, etc.), even if the object includes pixels with a luminance value of 0 among the pixels constituting the object, after being converted by the image processing of the input / output characteristics by the video control unit 1160 into an object with increased luminance values of the pixels in the low-luminance area, it is displayed on the display device 1 and converted into the floating-in-air video 3 by the optical system of the floating-in-air video display device 1000.

[0192] That is, the pixels constituting the object after the image processing of the input / output characteristics are converted to a state where pixels with a luminance value of 0 are not included, and then displayed on the display device 1, and are converted into the spatial floating image 3 by the optical system of the spatial floating image display device 1000.

[0193] In the image processing of FIG. 13B(2), as a method of performing the image processing of the input / output characteristics only on the image area of the object (character “panda” 1525), for example, in the image processing of the video control unit 1160, the background image layer and the character image layer in front of the background image layer are configured to be processed as separate layers so that the image processing of the input / output characteristics of FIG. 13B(2) can be performed on the character image layer, and the background image layer is not subjected to the image processing.

[0194] After that, by synthesizing these layers, as shown in FIG. 13B(2), only the character image is subjected to the image processing with the characteristic of raising the low-luminance area of the input image. As another method, after the character image layer and the background image layer are synthesized, the image processing of the input / output characteristics of FIG. 13B(2) may be configured to be performed only on the area of the character image.

[0195] In addition, the input / output video characteristics used in the video processing for raising the low-luminance area of the input / output characteristics with respect to the input video are not limited to the example of FIG. 13B(2). Any video processing that raises the low luminance may be used, such as so-called brightness adjustment. Alternatively, video processing for improving visibility may be performed by controlling the gain that changes the weighting of the retinex processing as disclosed in International Publication No. 2014 / 162533.

[0196] According to the image processing of FIG. 13B(2) described above, regarding the area where black is drawn among the areas for drawing images of characters, objects, etc., it is possible to make it recognizable to the user without melting into the black of the background, and it is possible to realize a more suitable display.

[0197] In the examples of FIGS. 13A and 13B, the problems and more suitable image processing are described by taking as an example a spatial floating image display device in which the background appears black (for example, the spatial floating image display device 1000 in FIGS. 4A to 4G, or the spatial floating image display device 1000 in FIGS. 4I and 4J in a state where the rear window is shielded from light). However, the said image processing is also effective in devices other than these spatial floating image display devices.

[0198] Specifically, in the spatial floating image display device 1000 in FIG. 4H or the spatial floating image display device 1000 in FIGS. 4I and 4J in a state where the rear window is not shielded from light, the background of the spatial floating image 3 is not black but the scenery behind the spatial floating image display device 1000 through the window. Also in this case, the problems described in FIGS. 13A and 13B exist in the same way.

[0199] That is, the portions of the image of the character "panda" 1525 that are drawn in black will blend into the scenery behind the spatial floating image display device 1000 through the window. Also in this case, by using the image processing in FIG. 13B(2), the portions of the image of the character "panda" 1525 that are drawn in black can be recognized separately from the scenery behind the spatial floating image display device 1000 through the window, and the visibility of the object is improved.

[0200] That is, by using the image processing in FIG. 13B(2), the area where the image of the character "panda" 1525, which is an object, is displayed can be recognized separately from the scenery behind the spatial floating image display device 1000 through the window, and it becomes possible to more preferably recognize that the character "panda" 1525, which is the object, is in front of the said scenery, and the visibility of the object is improved.

[0201] Also, in the spatial floating image display device 1000 of FIGS. 4K, 4L, and 4M, as described above, when another image (such as the image of the transmissive self-luminous image display device 1650 or the image of the second display device 1680) is displayed at a position having a different depth from the spatial floating image 3, the background of the spatial floating image 3 is not black but becomes the other image. Also in this case, the problems described with reference to FIGS. 13A and 13B similarly exist.

[0202] That is, the portion where black is drawn in the image of the character "panda" 1525, which is an object, will blend into the other image displayed at a position having a different depth from the spatial floating image 3. Also in this case, by using the image processing of FIG. 13B(2), the portion where black is drawn in the image of the character "panda" 1525, which is an object, can be recognized separately from the other image, and the visibility of the object is improved.

[0203] That is, by using the image processing of FIG. 13B(2), the area where the image of the character "panda" 1525, which is an object, is displayed can be recognized separately from the other image, and it becomes possible to more preferably recognize that the character "panda" 1525, which is the object, is in front of the other image, and the visibility of the object is improved.

[0204] An example of the video display process of this embodiment will be described with reference to FIG. 13C. FIG. 13C is an example of video display in which the spatial floating image 3 and a second image 2050, which is another image, are simultaneously displayed among the examples of video display of this embodiment. The second image 2050 may correspond to the display image of the transmissive self-luminous image display device 1650 in FIG. 4K or FIG. 4L. Also, the second image 2050 may correspond to the display image of the second display device 1680 in FIG. 4M.

[0205] That is, an example of the video display in FIG. 13C shows a specific example of the video display example of the spatial floating video display device 1000 in FIGS. 4K, 4L, and 4M. In the example of this figure, a bear character is displayed in the spatial floating video 3. The area other than the bear character in the spatial floating video 3 is black-displayed and becomes transparent as the spatial floating video. Also, the second image 2050 is a background image in which a plain, mountains, and the sun are drawn.

[0206] Here, in FIG. 13C, the spatial floating video 3 and the second image 2050 are displayed at different depths. When the user 230 visually recognizes the two videos of the spatial floating video 3 and the second image 2050 in the line-of-sight direction of the arrow 2040, the user 230 can visually recognize the videos in a state where the two videos overlap. Specifically, the bear character of the spatial floating video 3 can be seen superimposed in front of the background of the plain, mountains, and sun drawn in the second image 2050.

[0207] Here, since the spatial floating video 3 forms a real image in the air, when the user 230 slightly moves the viewing point, the user 230 can recognize the depths of the spatial floating video 3 and the second image 2050 due to the parallax. Therefore, the user 230 can obtain a stronger sense of spatial floating for the spatial floating video 3 while visually recognizing the two videos in an overlapping state.

[0208] An example of the video display process of this embodiment will be described with reference to FIG. 13D. FIG. 13D(1) is a view of the spatial floating video 3 seen from the line-of-sight direction of the user 230 among the examples of the video display of this embodiment in FIG. 13C. Here, a bear character is displayed in the spatial floating video 3. The area other than the bear character in the spatial floating video 3 is black-displayed and becomes transparent as the spatial floating video.

[0209] FIG. 13D(2) is a view of the second image 2050 seen from the line-of-sight direction of the user 230 among the examples of the video display of this embodiment in FIG. 13C. In the example of this figure, the second image 2050 is a background image in which a plain, mountains, and the sun are drawn.

[0210] FIG. 13D(3) is a diagram showing a state in which, among the examples of video display of the present embodiment in FIG. 13C, the second image 2050 and the spatially floating video 3 appear to overlap in the line-of-sight direction of the user 230. Specifically, in front of the background of the plain, mountains, and sun drawn in the second image 2050, the bear character of the spatially floating video 3 appears to overlap.

[0211] Here, when the spatially floating video 3 and the second image 2050 are displayed simultaneously, in order to more preferably ensure the visibility of the spatially floating video 3, it is desirable to pay attention to the balance of the brightness of the two videos. If the second image 2050 is too bright compared to the brightness of the spatially floating video 3, the displayed video of the spatially floating video 3 will become transparent, and the second image 2050, which is the background, will be strongly visible through the transparency.

[0212] Therefore, at least, the brightness per unit area of the spatially floating video 3 at the display position of the spatially floating video 3 should be greater than the brightness per unit area of the video light reaching the display position of the spatially floating video 3 from the second image 2050. The output of the light source of the spatially floating video 3, the display video brightness of the display device 1, the output of the light source of the display device for displaying the second image 2050, and the display video brightness of the said display device may be set.

[0213] Note that since this condition only needs to be satisfied when the spatially floating video 3 and the second image 2050 are displayed simultaneously, when switching from the first display mode in which only the second image 2050 is displayed without displaying the spatially floating video 3 to the second display mode in which the spatially floating video 3 and the second image 2050 are displayed simultaneously, the brightness of the second image 2050 may be reduced by lowering the output of the light source of the display device for displaying the second image 2050 and / or the display video brightness of the said display device. These controls may be realized by the control unit 1110 in FIG. 3 controlling the display device 1 and the display device for displaying the second image 2050 (the transmissive self-emitting video display device 1650 in FIG. 4K or FIG. 4L or the second display device 1680 in FIG. 4M).

[0214] In addition, when performing control to reduce the brightness of the second image 2050 in the switching from the above-described first display mode to the above-described second display mode, the brightness may be uniformly reduced for the entire screen of the second image 2050. Or, without uniformly reducing the brightness for the entire screen of the second image 2050, the portion where the object is displayed in the spatial floating image 3 is set to the state with the highest brightness reduction effect, and the brightness reduction effect may be gradually eased around it. That is, as long as the brightness reduction of the second image 2050 is realized only for the portion where the spatial floating image 3 is superimposed on the second image 2050 and visually recognized, ensuring the visibility of the spatial floating image 3 is sufficient.

[0215] Here, since the spatial floating image 3 and the second image 2050 are displayed at positions with different depths, when the user 230 slightly changes the viewing point, due to the parallax, the superimposition position of the spatial floating image 3 with respect to the second image 2050 changes. Therefore, in the switching from the above-described first display mode to the above-described second display mode, when reducing the brightness non-uniformly for the entire screen of the second image 2050, it is not desirable to sharply reduce the brightness based on the contour of the object displayed in the spatial floating image 3. It is desirable to perform the gradation processing of the brightness reduction effect, where the brightness reduction effect changes step by step according to the position as described above.

[0216] In the spatial floating image display device 1000 where the position of the object displayed in the spatial floating image 3 is approximately at the center of the spatial floating image 3, the position with the highest brightness reduction effect in the gradation processing of the brightness reduction effect may be set to the center position of the spatial floating image 3.

[0217] According to the video display processing of this embodiment described above, the user 230 can more suitably visually recognize the spatial floating image 3 and the second image 2050.

[0218] When displaying the floating image 3 in space, it may be controlled so as not to display the second image 2050. Since the visibility of the floating image 3 in space is improved when the second image 2050 is not displayed, this control is suitable for a floating image display device 1000 in a use where the user must surely visually recognize the floating image 3 in space when the floating image 3 in space is displayed.

[0219] <Example 2> As Example 2 of the present invention, an example of another configuration example of a floating image display device in space will be described. The floating image display device according to the present example is obtained by changing the optical system stored in the floating image display device, which was described in Example 1, to the optical system shown in FIG. 14(1) or FIG. 14(2). In this example, the differences from Example 1 will be described, and the description of the same configuration as in Example 1 will be omitted. In the following description of this example, a predetermined polarization and the other polarization are polarizations of polarization waves having a phase difference of 90° from each other.

[0220] FIG. 14(1) is an example of the optical system and the optical path according to the present example. The optical system shown in FIG. 14(1) is obtained by bringing the display device 1 closer to the polarization separation member 101B in the optical system of FIG. 2C, and the entire optical system is configured to be more compact. In FIG. 14(1), the description of the configuration denoted by the same reference numeral as in FIG. 2C will be omitted for the sake of brevity.

[0221] In FIG. 14(1), similar to FIG. 2C, the video light of a predetermined polarization (P polarization in the figure) emitted from the display device 1 travels in the vertical direction from the video display surface of the display device 1. Here, the polarization separation member 101B selectively transmits a predetermined polarization (P polarization in the figure) emitted from the display device 1 and reflects the other polarization (S polarization in the figure), similar to FIG. 2C.

[0222] Therefore, the video light of a predetermined polarization (P-polarization in the figure) that travels in the vertical direction from the video display surface of the display device 1 passes through the polarization separation member 101B and reaches the retroreflective plate 2 to which the λ / 4 plate 21 is attached. The video light that is retroreflectively reflected by the retroreflective plate 2 and travels again toward the polarization separation member 101B has passed through the λ / 4 plate 21 twice, and thus has been converted from the predetermined polarization (P-polarization in the figure) at the time of emission from the display device 1 to the other polarization (S-polarization in the figure). Since the video light that travels again toward the polarization separation member 101B is the other polarization (S-polarization in the figure), it is reflected by the polarization separation member 101B toward the position where the user should be. The traveling direction of the video reflected by the polarization separation member 101B is determined based on the angle at which the polarization separation member 101B is disposed.

[0223] In the example of FIG. 14(1), the video light that travels toward the polarization separation member 101B is reflected at a right angle by the polarization separation member 101B and travels as shown in the figure. The video light reflected by the polarization separation member 101B forms a spatial floating image 3A. The spatial floating image 3A can be suitably visually recognized by the user from the direction of arrow A.

[0224] Here, due to the characteristics of the retroreflective reflection by the retroreflective plate 2, the optical path length from when the video light emitted from the display device 1 reaches the retroreflective plate 2 and the optical path length from when the video light emitted from the retroreflective plate 2 reaches the formation position of the spatial floating image 3A are in an equal relationship. Due to this relationship, the formation position of the spatial floating image 3A in the traveling direction of the video light reflected by the polarization separation member 101B is determined.

[0225] In the example of FIG. 14(1), the display device 1, the polarization separation member 101B, and the retroreflective plate 2 are arranged so as to be closer than in the example of FIG. 2C. This realizes a more compact configuration of the entire optical system. However, the amount by which the spatial floating image 3A protrudes from the optical system of FIG. 14(1) is not so large. For example, as one index of the amount by which the spatial floating image 3A protrudes from the optical system, the distance (L1 in the example of FIG. 14(1)) from the position where the light beam at the center portion of the video light is reflected by the polarization separation member 101B to the position where the video light forms the spatial floating image 3A is shown in the figure.

[0226] Regarding the polarization design in the optical system of Fig. 14(1), the characteristics of P-polarized light and S-polarized light may be interchanged. Specifically, a predetermined polarization of the video light emitted from the display device 1 may be set as S-polarized light, and regarding the reflection characteristics of the polarization separation member 101B, the characteristics of P-polarized light and S-polarized light may be interchanged. In this case, although both the P-polarized light and S-polarized light shown in the figure are reversed, the optical design such as the optical path can be realized in exactly the same way.

[0227] Next, Fig. 14(2) shows another example of the optical system and the optical path according to this embodiment. The optical system of Fig. 14(2) modifies the configuration in the optical system of Fig. 14(1) in order to make the amount of the spatial floating image jumping out from the optical system larger while realizing the same compactness as the optical system of Fig. 14(1). In Fig. 14(2), for the configurations labeled with the same reference numerals as in Fig. 14(1), repeated detailed descriptions will be omitted.

[0228] In Fig. 14(2), similar to Fig. 14(1), the video light of a predetermined polarization (P-polarized light in the figure) emitted from the display device 1 travels in the vertical direction from the video display surface of the display device 1. Here, the polarization characteristics of the polarization separation member 101B are arranged 90 degrees differently from those in Fig. 14(1). The video light of a predetermined polarization (P-polarized light in the figure) that has traveled in the vertical direction from the video display surface of the display device 1 passes through the polarization separation member 101B.

[0229] Here, different from Fig. 14(1), instead of the retroreflector 2 with the λ / 4 plate 21 attached, a specular reflector 4 with the λ / 4 plate 21B attached is arranged at the position where the video light passes through the polarization separation member 101B. Here, the reflection in the specular reflector 4 is specular reflection (also called regular reflection), not retroreflective reflection.

[0230] Therefore, the video light transmitted through the polarization separation member 101B is specularly reflected by the specular reflection plate 4 to which the λ / 4 plate 21B is attached. The video light that is specularly reflected by the specular reflection plate 4 and travels again toward the polarization separation member 101B is converted from a predetermined polarization (P polarization in the figure) at the time of emission from the display device 1 to the other polarization (S polarization in the figure) by passing through the λ / 4 plate 21 twice. Since the video light that has traveled again toward the polarization separation member 101B is the other polarization (S polarization in the figure), it is reflected by the polarization separation member 101B.

[0231] Here, since the orientation of the polarization separation member 101B in Fig. 14(2) is different from that in Fig. 14(1), the video light reflected by the polarization separation member 101B travels in the direction opposite to the position where the user should be. A retroreflective plate 2 to which the λ / 4 plate 21C is attached is arranged at the destination where the video light reflected by the polarization separation member 101B travels. The video light is retroreflectively reflected by the retroreflective plate 2. The video light that is retroreflectively reflected by the retroreflective plate 2 and travels again toward the polarization separation member 101B is converted from the other polarization (S polarization in the figure) back to the predetermined polarization (P polarization in the figure) by passing through the λ / 4 plate 21C twice.

[0232] Since the video light that has traveled again toward the polarization separation member 101B is the predetermined polarization (P polarization in the figure), it passes through the polarization separation member 101B and travels directly toward the position where the user should be. The video light passing through the polarization separation member 101B forms a spatial floating image 3B. The spatial floating image 3B can be suitably viewed by the user from the direction of arrow A.

[0233] Here, also in Fig. 14(2), similar to Fig. 14(1), due to the characteristics of the retroreflective reflection by the retroreflective plate 2, the optical path length from the display device 1 to the point where the video light emitted from the display device 1 reaches the retroreflective plate 2 and the optical path length from the retroreflective plate 2 to the point where the video light emitted from the retroreflective plate 2 reaches the formation position of the spatial floating image 3B are equal. Based on this relationship, the formation position of the spatial floating image 3B in the traveling direction of the video light transmitted through the polarization separation member 101B is determined.

[0234] In Fig. 14(2), the optical path length from the display device 1 to the point where the video light emitted from the display device 1 reaches the retroreflector 2 is longer than the optical path length from the display device 1 to the point where the video light emitted from the display device 1 reaches the retroreflector 2 in Fig. 14(1). In the optical system of Fig. 14(2), an optical path that reciprocates between the polarization separation member 101B and the specular reflector 4, which does not exist in the optical system of Fig. 14(1), is added to the optical path length from the display device 1 to the point where the video light reaches the retroreflector 2.

[0235] As a result, in the optical system of Fig. 14(2), the distance (L2 in the example of Fig. 14(2)) from the position where the light rays in the central part of the video light pass through the polarization separation member 101B to the position where the video light forms the spatial floating image 3B is much longer than the distance (L1 in the example of Fig. 14(1)) from the position where the light rays in the central part of the video light are reflected by the polarization separation member 101B to the position where the video light forms the spatial floating image 3A in the optical system of Fig. 14(1).

[0236] Regarding the polarization design in the optical system of Fig. 14(2), the characteristics of P-polarized light and S-polarized light may be interchanged. Specifically, a predetermined polarization of the video light emitted from the display device 1 may be set as S-polarized light, and the reflection characteristics of the polarization separation member 101B may have the characteristics of P-polarized light and S-polarized light interchanged. In this case, although both the illustrated P-polarized light and S-polarized light are reversed, the optical design such as the optical path can be realized in exactly the same way.

[0237] According to the optical systems of Figs. 14(1) and 14(2) in the second embodiment of the present invention described above, a more compact optical system can be realized. In particular, according to the optical system of Fig. 14(2), it is possible to increase the amount by which the spatial floating image protrudes from the optical system while being a more compact optical system.

[0238] In addition, when incorporating the optical system of FIG. 14(1) or FIG. 14(2) into the spatial floating image display device, it can be realized by replacing the optical system in the spatial floating image display device described in Example 1 with the optical system of FIG. 14(1) or FIG. 14(2). Specifically, the optical system of FIG. 14(1) may be replaced with the optical system of the spatial floating image display device of FIGS. 4E, 4F, 4G, 4H, 4I, 4J, 4K, 4L, or 4M. In this case, since the optical system becomes more compact, it is possible to make the housing of the spatial floating image display device in each figure smaller.

[0239] Specifically, the optical system of FIG. 14(2) may also be replaced with the optical system of the spatial floating image display device of FIGS. 4E, 4F, 4G, 4K, or 4L. In this case, it is possible to increase the amount by which the spatial floating image protrudes from the optical system. Also, since the optical system becomes more compact, it is possible to make the housing of the spatial floating image display device in each figure smaller.

[0240] <Example 3> As Example 3 of the present invention, a configuration example of the spatial floating image display device will be described. As the basic configuration, the configurations of the respective figures described in Example 1 and Example 2 can be similarly applied. In this example, the differences from Example 1 and Example 2 will be mainly described, and repeated descriptions of the same configurations as in Example 1 and Example 2 will be omitted.

[0241] The spatial floating image display device of this Example 3 is provided as one component in a predetermined device such as a vending machine. This spatial floating image display device is provided as a user interface (UI) for operations / inputs such as product selection in a predetermined device. In other words, this spatial floating image display device functions as an operation unit, an input unit, a display unit, and a UI unit. In Example 3, the case where the spatial floating image display device is applied to a vending machine is shown. However, it is not limited to this, and it can be similarly applied to devices that require selection operations, such as public facilities.

[0242] [Problems, etc.] In devices such as vending machines, there are many users who do not want to directly touch shared objects that are touched by an unspecified number of people, such as physical buttons or display buttons on a touch panel. That is, there is a demand for a non-contact operation function for operating objects such as buttons without contact. Providing such a non-contact operation function is one of the features of the spatial floating image display device of each embodiment.

[0243] In Embodiment 3, as an operation unit of a vending machine, a UI using a spatial floating image by a spatial floating image display device is applied. Thereby, a non-contact operation function is provided that allows a user to select a purchased product by non-contact operation.

[0244] In addition, when implementing a spatial floating image display device (in other words, an air display) in a device such as a vending machine, miniaturization suitable for the device such as a vending machine is also required. Further, when implementing a spatial floating image display device as an operation unit and a UI in a device such as a vending machine, it is also required that the user operation of the spatial floating image display device, that is, the air operation on the spatial floating image, is simpler and easier to use. In Embodiment 3, in order to satisfy these requirements, a spatial floating image display device is mounted on the main body of the vending machine, and a suitable UI using a spatial floating image is provided.

[0245] The vending machine equipped with the spatial floating image display device in Embodiment 3 implements a UI using a spatial floating image as an operation unit for predetermined operations such as product selection and determination. By receiving and detecting an air operation on the image of this UI, operations such as product selection and determination are realized. In Embodiment 3, the image of this UI is made into an image like a wheel described later, so as to prompt the user to perform operations in the vertical and horizontal directions (plane directions) corresponding to the two-dimensional plane of the spatial floating image. In Embodiment 3, in particular, according to the air operation on the image of this UI, it is possible to select one product selection button from a plurality of product selection buttons in the product display section of the vending machine. Also, in Embodiment 3, control such as lighting the selected product selection button is performed.

[0246] [Commodity vending machine] FIG. 15 shows a configuration example of a commodity vending machine 3000 provided with the spatial floating video display device 1000 of Embodiment 3 as an operation unit 1510. The operation unit 1510 may be provided with any one of the spatial floating video display devices 1000 described in Embodiment 1 or Embodiment 2. Hereinafter, in various descriptions of the commodity vending machine of the present embodiment, the spatial floating video display device 1000 which is the operation unit 1510 has, for example, the configuration shown in FIG. 3. The operations of the respective components of the spatial floating video display device 1000 which is the operation unit 1510 are controlled by the control unit 1110 as described in FIG. 3. The video display operation of the spatial floating video display device 1000 which is the operation unit 1510 is displayed on the display device 1 based on the control of the video control unit 1160 as described in FIG. 3. The operation detection operation of the spatial floating video display device 1000 which is the operation unit 1510 is based on the sensing result by the air operation detection sensor 1351 as described in FIG. 3. The air operation detection unit 1350 performs an operation detection process and generates control information based on the operation detection. Since the functions and operations of the other configurations shown in FIG. 3 are as described in Embodiment 1 or Embodiment 2, repeated descriptions are omitted. FIG. 15 shows an x-z plane view when the front surface 3591 of the housing 3590 of the main body 3500 of the commodity vending machine 3000 is viewed from the front by the user. The main body 3500 of the commodity vending machine 3000 has a generally rectangular housing 3590. On the front surface 3591 of the housing 3590, a commodity display section 1501, commodity selection buttons 1502, etc. are provided. Also, on the front surface 3591, a commodity outlet 3592, a coin / bill insertion section 3593, a settlement device 1702, etc. are provided.

[0247] In the present embodiment, on the front surface 3591, an operation unit 1510 by the spatial floating video display device 1000 is provided in a region on the right side of the commodity display section 1501, for example, at an upper position such as the settlement device 1702. The position where this operation unit 1510 is arranged is an example. The operation unit 1510 is, in other words, an operation unit and an operation device.

[0248] In the operation unit 1510, in the rectangular opening 1515 on the front surface 3591, the spatial floating image 3 by the spatial floating image display device 1000 is formed and displayed. The display range 3R of the spatial floating image 3 is provided, for example, in accordance with the size of the opening 1515. This spatial floating image 3 includes the image 1520 of the UI for receiving the aerial operation by the user, the product purchaser. This image 1520 is, although will be described in detail later, an image like a wheel that prompts the planar operation (including the operation in the x direction and the operation in the z direction) in the x-z plane of the spatial floating image 3. In the example of FIG. 15, this image 1520 is an image / picture represented by a three-dimensional sphere. This image 1520 is, in other words, an operation image, a UI image, a wheel image, a sphere image, an object image, and the like.

[0249] In the third embodiment, the user looks at the image 1520 of this operation unit 1510 and performs an aerial operation, in other words, a touch operation, on this image 1520 with the finger 231 (for example, one finger). This aerial operation includes at least the operations in the x direction (in other words, the horizontal direction, the left-right direction) and the z direction (in other words, the vertical direction, the up-down direction) shown in the figure. The two arrows shown in the image 1520 represent, for explanatory purposes, the operations in the x direction and the z direction and the state of rotation of the wheel image 1520 according to the operations. The spatial floating image display device 1000 detects the aerial operation on this image 1520. This detection includes the detection regarding the movement, in other words, the displacement, of the touch / contact position of the finger 231 in the x direction and the z direction.

[0250] Based on the information of the detection of the aerial operation on the image 1520 by the spatial floating image display device 1000 of the operation unit 1510, the vending machine 3000 selects one from the plurality of product selection buttons 1502 in the product posting unit 1501 and performs control such as lighting the selected product selection button 1502. Thereby, it is possible to convey to the user that the product associated with the lit product selection button 1502 is selected as the purchase target.

[0251] [Vending Machine: Configuration Example] FIG. 16 shows a configuration example of the merchandise vending machine 3000. The merchandise vending machine 3000 has a merchandise vending machine main body 3500 and an operation unit 1510 by the spatial floating image display device 1000. The merchandise vending machine main body 3500 includes a control unit 3501, a memory 3502, a non-volatile memory 3503, a communication unit 3504, a communication interface 3505, a data storage 3506, a position information acquisition unit 3507, a power supply 3508, an external power supply input interface 3509, a merchandise vending machine function unit 3510, etc., and these are interconnected by an architecture such as a bus.

[0252] The control unit 3501 includes a processor and functions as a controller. The control unit 3501 controls each part and the whole of the merchandise vending machine 3000. Various data and information processed by the control unit 3501 are stored in the memory 3502. Various data and information handled by the control unit 3501 are held in the non-volatile memory 3503. Various data and information such as programs and video data are stored in the data storage 3506. The communication unit 3504 has, for example, a wireless communication interface implemented and can communicate with external devices. The communication interface 3505 is a communication interface with the spatial floating image display device 1000 and communicates with the spatial floating image display device 1000. The position information acquisition unit 3507 has a GPS receiver etc. implemented and can acquire position information. The power supply 3508 supplies the necessary power to each part of the merchandise vending machine 3000 based on an external power supply input from the external power supply input interface 3509.

[0253] The spatial floating image display device 1000, which is the operation unit 1510, communicates with the communication interface 3505 of the vending machine main body 3500 via the communication unit 1132 under the control of the control unit 1110 in FIG. 3. Through this communication, various control information such as control information based on the detection of air operations can be transmitted from the spatial floating image display device 1000 to the vending machine main body 3500. Note that the external power input interface 1111 in FIG. 3 of the spatial floating image display device 1000, which is the operation unit 1510, is connected to the power supply 3508 of the vending machine main body 3500 in FIG. 16. Thereby, a power supply for operating the spatial floating image display device 1000, which is the operation unit 1510, is ensured.

[0254] [Vending Machine Function Unit] FIG. 17 shows a configuration example of the vending machine function unit 3510 of the vending machine main body 3500. The vending machine function unit 3510 is a component for realizing the functions of the vending machine 3000. Known technologies can be applied to the vending machine function unit 3510. The vending machine function unit 3510 includes a product display section / display 1501, an image processing unit 1701, an electronic information medium reader / writer 1702, an imaging unit 1703, an audio output unit 1704, an audio input unit 1705, a coin / bill processing unit 1706, a product storage unit 1707, a product vending mechanism 1708, etc., and these are interconnected by an architecture such as a bus.

[0255] The product display section / display 1501 is a product display section or a display, which combines the case of being composed of a product display section and the case of being composed of a display in one illustration, and either implementation is acceptable. When it is composed of a product display section, an actual product, a model, etc. are displayed, and there is no display. When it is composed of a product display section, it also includes a lighting device for illuminating the displayed product, etc. When it is composed of a display, an image of a product, etc. is displayed on the screen of the display. The display may be a touch panel. When it is composed of a product display section, the product selection button 1502 is composed of a physical operation button and a light-emitting section. When controlling the product selection button 1502 to light up, it is controlled to make the light-emitting section emit light. When it is composed of a display, the product selection button 1502 may be displayed as an image of a button on the screen of the display by means of a touch panel, for example, or the product image itself may function as a button.

[0256] When the product display section / display 1501 is composed of a display (such as a touch panel), the video processing unit 1701 processes video information / video data, creates data / signals for displaying a video on the screen of the display, and causes the video to be displayed on the screen of the display based on the data / signals.

[0257] The electronic information medium reader / writer 1702 is a device that reads and writes to an electronic information medium and functions as a payment device 1702. The payment device 1702 reads and writes information, for example, by means of the IC card of the user 230 or the NFC (Near Field Communication) of a smartphone. In this embodiment, a payment device 1702 by the electronic information medium reader / writer 1702 is provided as part of the product vending machine functional unit 3510. In this embodiment, the cooperation and control between the operation unit 1510 and the spatial floating video display device 1000 are mainly performed by the control unit 3501. However, it is not limited to this. For example, the payment device 1702 may perform the cooperation and control with the spatial floating video display device 1000.

[0258] The imaging unit 1703 captures an image of the vicinity of the housing 3590 in FIG. 15, for example, the vicinity of the front surface 3591, using a camera. Thereby, the imaging unit 1703 also functions as a human presence sensor that detects a user in the vicinity of the front surface 3591. In addition to the imaging unit 1703, various sensors such as an infrared sensor may be provided. The imaging unit 1703 may be realized as a part of the payment device 1702. User authentication or the like using the imaging unit 1703 may be performed. One or both of the imaging unit 1703 of the vending machine 3000 and the imaging unit 1180 (FIG. 3) of the spatial floating image display device 1000 may be used, or they may be integrated into one.

[0259] The voice output unit 1704 outputs a predetermined voice from a speaker. The voice input unit 1705 inputs the user's voice through a microphone. The coin / bill processing unit 1706 processes the coins or bills inserted by the user from the coin / bill insertion unit 3593. The product storage unit 1707 stores products such as beverages. The product vending mechanism 1708 vends the purchased product from the product storage unit 1707 to the product outlet 3592.

[0260] The communication interface 3505 communicates with the spatial floating image display device 1000 of the operation unit 1510. Control information (hereinafter referred to as detection information / control information) based on the detection of an air operation from the spatial floating image display device 1000, which is the operation unit 1510, is transmitted from the spatial floating image display device 1000, which is the operation unit 1510, to the control unit 3501 via the communication. In various descriptions related to the vending machine 3000 of this embodiment, the user's air operation at the operation unit 1510 is reflected in the operation or processing of the vending machine functional unit 3510 via the communication and the control of the control unit 3501.

[0261] Based on the cooperation through the communication between the operation unit 1510 and the spatial floating image display device 1000, the control unit 3501 controls the product selection button 1502 of the product posting unit 1501, etc. For example, based on the detection information / control information from the operation unit 1510, the control unit 3501 transmits a signal 1711 for control to the product selection button 1502. The signal 1711 is, for example, a signal for lighting up the product selection button 1502 in the selected state by the user's air operation, and is a signal for instructing on / off of lighting, etc. Based on the signal 1711, the light emitting part of the product selection button 1502 in the selected state emits light.

[0262] [Vending Machine: Mounting of Spatial Floating Image Display Device] FIG. 18 shows an example of the mounting of the spatial floating image display device 1000 in the vending machine main body 3500 in a side view. This example is an example when based on the optical system of FIG. 2D. The configuration example of the spatial floating image display device 1000 is basically the same as the configuration examples of FIGS. 4O and 4P, and the difference is that the overall arrangement direction is rotated, and the emission direction of the image light forming the spatial floating image 3 is in the horizontal direction (-y direction).

[0263] In the housing 3590, image light is emitted downward (-z direction) from the display device 1 arranged on the upper side. The image light is retroreflected by the retroreflective plate 5 arranged obliquely in the housing 3590, and the reflected light is emitted forward (-y direction). The reflected light forms the spatial floating image 3, which is a real image, at the opening 1515 of the front surface 3591 of the housing 3590.

[0264] In the example of FIG. 18, the emission direction of the image light is in the horizontal direction (-y direction), and a vertically arranged spatial floating image 3 (display range 3R) is formed according to the position of the front surface 3591, but it is not limited to this.

[0265] There is a limit to the length in the depth direction (y direction) of the housing 3590 of the vending machine body 3500. In this embodiment, the spatial floating image display device 1000 mounted in the housing 3590 is a spatial floating image display device 1000 that has been miniaturized in accordance with the limit on the length in the depth direction (y direction) of the housing 3590 and has a relatively small depth length. In response to the miniaturization, the display range 3R of the spatial floating image 3 by this spatial floating image display device 1000 becomes a screen with a relatively small size. The image 1520 of the wheel is displayed within a screen of this size.

[0266] [Vending Machine: Product Display Section] FIG. 19 shows a configuration example of the product display section 1501 in the vending machine body 3500 in a side view. When a user operates the product selection button 1502 as an example of the operation of the vending machine, the following configuration and operations are involved. The user 230 views the product display section 1501 and selects and operates, that is, presses as a direct contact operation, the product selection button 1502 corresponding to the product to be purchased. In response to the pressing of the product selection button 1502, the product selection button 1502 lights up. This lighting indicates the selected state of the product. Thereafter, the user 230 operates, for example, the payment device 1702 or the coin / bill insertion unit 3593 to purchase and acquire the product.

[0267] Note that the lighting method of the product selection button 1502 may vary depending on the implementation details of the vending machine 3000. In other examples, the following may also be applicable. When the user 230 inserts coins / bills, all the product selection buttons 1502 of the products that can be purchased within the inserted amount range will have a predetermined lighting (referred to as the first lighting). From these product selection buttons 1502, the user 230 selects and operates, that is, presses, one product selection button 1502 corresponding to the product for which the purchase is to be determined. In response to the pressing of the product selection button 1502, the product is purchased. In response to the pressing of the product selection button 1502, the product selection button 1502 may have a predetermined lighting (referred to as the second lighting).

[0268] In contrast, in the third embodiment, the following configurations and operations are further available, and it is not necessary to press the product selection button 1502 as a direct contact operation. The user 230 checks a product to be purchased by looking at the product display section 1501. The user 230 selects the product selection button 1502 corresponding to the desired product by performing an air operation on the image 1520 of the wheel of the operation section 1510. In response to this selection operation, the product vending machine 3000, particularly the control section 3501, lights up the product selection button 1502. This lighting indicates the selected state. Thereafter, the user 230 determines to purchase the product corresponding to the product selection button 1502 in the selected state by operating, for example, a determination button described later. Thereafter, the user 230 operates, for example, the payment device 1702 or the coin / bill insertion section 3593 to purchase and acquire the product.

[0269] In another example, the following may also be applicable. When the user 230 inserts coins / bills, all the product selection buttons 1502 of the products that can be purchased within the inserted amount range are lit in a predetermined manner (first lighting). The user 230 selects one product selection button 1502 corresponding to the desired product to be purchased from the product selection buttons 1502 in the state of the first lighting by performing an air operation on the image 1520 of the wheel of the operation section 1510. In response to this selection operation, the product vending machine 3000, particularly the control section 3501, controls the product selection button 1502 to be lit in a predetermined manner (second lighting). The second lighting indicates the selected state of the product. Thereafter, the user 230 operates, for example, the payment device 1702 or the coin / bill insertion section 3593 to purchase and acquire the product.

[0270] [Product vending machine: Operation section (1)] Figure 20A shows a configuration example of the operation unit 1510 in Embodiment 3 in a side view. The configuration example in Figure 20A corresponds to the configuration example in Figure 18. The front surface 3591 of the housing 3590 has an opening 1515 so as to form a recess on the back side (+y direction). Based on the video light A1 traveling in the horizontal direction (-y direction) emitted from the spatial floating video display device 1000, a spatial floating video 3 is formed at a position aligned with the front surface 3591 in the y direction at the opening 1515. In this example, the spatial floating video 3 is arranged to stand in the vertical direction (z direction). The display range 3R of the spatial floating video 3 forms the x-z plane. The user 230 can more preferably visually recognize this spatial floating video 3 in the line-of-sight direction (+y direction) indicated by the arrow A corresponding to the direction of the video light A1.

[0271] In Figure 20A, a front view (x-z plane view) when the spatial floating video 3 is viewed from the front is also shown. The user 230 looks at the video 1520 of the wheel in the spatial floating video 3 and performs an in-air operation such as a touch or a slide with the finger 231. Thereby, product selection and purchase decision are possible. Therefore, in this Embodiment 3, an operation in which the user 230 directly touches the product selection button 1502, which is a physical button of the product posting unit 1501 as shown in Figure 19, that is, pressing down is unnecessary.

[0272] Of course, depending on the user 230 and the situation, there may be cases where a contact operation of directly pressing down the physical product selection button 1502 (Figure 19) is also acceptable. In this case, as in the prior art, product selection and the like are possible using the contact operation. The mechanism of the contact operation of physical buttons such as the product selection button 1502 can be realized in the same manner as in the prior art and coexists with the mechanism of the in-air operation of the spatial floating video 3 in this embodiment. In this Embodiment 3, a contact operation in which the user 230 directly presses down the product selection button 1501 is also allowed.

[0273] Also, depending on the user 230 or the situation, both the air operation of the spatial floating image 3 and the contact operation of physical buttons such as the product selection button 1501 may be used in combination. For example, the user 230 first selects a product selection button 1502 of a certain product by an air operation of the wheel image 1520. Then, when the user 230 looks at the product posting section 1501 and changes to purchasing another product, the user 230 may directly perform a contact operation on the product selection button 1502 corresponding to the other product. By this contact operation, the other product becomes the selected state. In another example, the user 230 first makes a certain product in the selected state by directly performing a contact operation on the product selection button 1502 of the product. Then, when the user 230 changes to purchasing another product, the user 230 can select the product selection button 1502 of the other product by an air operation of the wheel image 1520.

[0274] [Vending Machine: Operation Unit (2)] FIG. 20B shows, as a modification, a configuration example of the operation unit 1510 in a side view. In the example of FIG. 20B, the opening 1515 of the operation unit 1510 is provided with a frame 1530, which is a physical frame constituting the upper, lower, left, and right sides of the opening 1515, so as to protrude slightly forward (-y direction) with respect to the front surface 3591 of the housing 3590. The distance from the front surface 3591 to the spatial floating image 3 is indicated by the distance d20B. This frame 1530 is a portion surrounding the upper, lower, left, and right sides of the display range 3R of the spatial floating image 3. This frame 1530 has a width of d20B or more in the y direction. The spatial floating image 3 is formed at a position that protrudes forward from the front surface 3591 by a distance d20B in the space within this frame 1530. The user 230 performs an air operation on the image 1520 of the spatial floating image 3 so as to put the finger 231 into this frame 1530.

[0275] [Vending Machine: Operation Unit (3)] FIG. 20C shows, as a modified example, a configuration example of the operation unit 1510 in a side view. In the example of FIG. 20C, a frame 1530 as in FIG. 20B is not provided. The spatial floating image 3 is formed at a position that is distanced d20C from the front surface 3591 of the housing 3590 toward the front side. Also, as shown in the drawing, a protective glass plate 1540, which is a transparent member, is provided in a region corresponding to the opening 1515 on the front surface 3591. In addition to the protective glass plate 1540, other optical elements or the like may be provided. The image light A1 emitted from the spatial floating image display device 1000 passes through the protective glass plate 1540 and forms the spatial floating image 3 at the position shown in the drawing.

[0276] [Product vending machine: Operation unit (4)] FIG. 20D shows, as a modified example, a configuration example of the operation unit 1510 in a side view. The example of FIG. 20D is an example in which the spatial floating image display device 1000 is mounted in the same manner as the configuration example as in FIG. 4P. In the example of FIG. 20D, with respect to the front surface 3591 of the housing 3590, the image light A1 is emitted obliquely upward, so that the optical axis of the spatial floating image 3 is arranged to face obliquely upward. The image light A1 forms the spatially floating image 3 arranged obliquely as shown in the drawing at the obliquely oriented opening 1515. The user 230 can more suitably visually recognize this spatial floating image 3 in the line-of-sight direction indicated by the arrow B that faces obliquely downward corresponding to the optical axis of the spatial floating image 3.

[0277] [Image of the wheel] FIGS. 21 and 22 show display configuration examples of the wheel image 1520 in the spatial floating image 3 formed in the opening 1515 of the operation unit 1510. Any of the display configuration examples may be adopted. The image 1520 in state A is represented as a simple sphere as a three-dimensional object. This sphere is displayed in a predetermined color (for example, red). This sphere is also given luster and shadow so as to give a three-dimensional effect. Not limited to this, the sphere of the image 1520 may have a plurality of colors, patterns, etc. attached to the sphere surface.

[0278] In state B, in the display range 3R of the spatial floating image 3, in addition to the image 1520 by the sphere, a character image 2102 is additionally displayed. In this example, the character image 2102 is displayed separately above the image 1520 by the sphere, but the character image 2102 may be displayed overlapping the sphere. The details of the character image 2102 are arbitrary and may be an icon, an animation, or the like. The character image 2102 may be, for example, an image of a guide such as "Please operate vertically and horizontally" or "Please select a product".

[0279] In the image 1520 in state C, lines 2103 such as latitude and longitude are additionally displayed on the surface of the sphere so that the state of rotation can be easily understood. In response to an air operation by the user, the image 1520 of the sphere rotates, and the display state of the lines 2103 such as latitude and longitude also changes according to the state of the rotation.

[0280] In the image 1520 in state D, an arrow image 2104 representing the rotation of the sphere is additionally displayed with respect to the sphere. The arrow image 2104 may be displayed in advance before the air operation by the user. For example, the illustrated arrow image 2104 represents an operation in the x direction. Also, the arrow image 2104 may not be displayed first, and the arrow image 2104 may be displayed when the user performs an air operation. In that case, the arrow image 2104 represents the direction of the operation performed by the user. Not limited to the arrow image, other effect images for expressing a rotation operation or the like may be displayed. In another example, the image 1520 may be displayed in a first color when no air operation is performed and in a different second color when an air operation is performed.

[0281] Note that the background of the image 1520 (for example, a sphere) in the spatial floating image 3 is an arbitrary display and may be roughly black or white. It may be configured so that the image 1520 stands out against the background. Also, although the display range 3R of the spatial floating image 3 is a square, of course, it is not limited to this and is possible.

[0282] In FIG. 22, the video 1520 in state E is displayed as a three-dimensional object composed of a disk (first wheel) rotatable in the x direction and a disk (second wheel) rotatable in the z direction. The first wheel indicates that it can be rotated in the x direction, and the second wheel indicates that it can be rotated in the z direction. It is not limited to a disk shape and may be a ring shape or the like.

[0283] The video 1520 in state F is displayed as a two-dimensional circle instead of a three-dimensional sphere, and as an object with arrow images overlaid in the up, down, left, and right directions. These arrow images indicate that operations in the x direction and the z direction are possible. Also, these arrow images may be four arrow images separated in the four directions of up, down, left, and right.

[0284] The video 1520 in state G is displayed as a two-dimensional cross-shaped object. This cross shape indicates that operations in the x direction and the z direction are possible. Also, the cross-shaped image may be four images separated into four parts in the four directions of up, down, left, and right. In the case of the images in state F or state G, when user 230 performs an air operation on one of the four parts, it may be determined as an operation in the direction corresponding to that part.

[0285] The video 1520 in state H is displayed as an object with only the disk (first wheel) rotatable in the x direction. For example, when only operations in the x direction are required, the video 1520 may be displayed in such a way as to indicate acceptance / encouragement of operations in only the x direction. Similarly, it may be displayed with only the disk (second wheel) rotatable in the z direction.

[0286] The video 1520 in state I is, as a modification example, composed of numeric buttons. This video 1520 is not a video that accepts operations in the x or z directions, but a button object for designating / inputting numbers corresponding to product numbers. These numeric buttons may be provided as a plurality of buttons that correspond one-to-one with the product selection buttons 1502, or may be provided as a predetermined number of general-purpose buttons, for example, 10 buttons from 0 to 9. What is shown in the figure is the case of providing 10 buttons from 0 to 9. By the user 230 selecting from these numeric buttons and performing an in-air operation (touch operation) one or more times, it becomes possible to select / designate a desired product number. For example, when the user 230 successively touches the "1" button and the "2" button, product number 12 can be selected / designated. Also, in this modification example, an image representing the state of the number selected / designated by the in-air operation (touch operation) of the numeric button video 1520 may be further displayed within the spatial floating video 3. Additionally, a numeric cancellation button, a numeric determination button, etc. may be separately provided.

[0287] [Vending Machine: Sequence] FIG. 23 shows a configuration example of a sequence regarding processing and operations between the control unit 3501 of the vending machine 3000 and the spatial floating video display device 1000, which is the operation unit 1510, and the user 230 in the system of Example 3. The main subject of the control processing in the spatial floating video display device 1000 is a video processing unit such as the control unit 1110 or the video control unit 1160 in FIG. 3. Step S1 is the initial state of the system of the vending machine 3000. In this example, in the initial state, the spatial floating video 3 is not displayed on the operation unit 1510. The control unit 3501 instructs the spatial floating video display device 1000 to make the spatial floating video 3 non-displayed. In a modification example, the spatial floating video 3 may be displayed on the operation unit 1510 even in the initial state.

[0288] Step S2 is the detection that user 230 has approached the product vending machine 3000. The product vending machine 3000 detects this proximity using, for example, the camera of imaging unit 1703. This detection may also be the detection of any user operation such as the contact operation of a physical button. Based on this detection, in step S3, the control unit 3501 instructs the spatial floating image display device 1000, which is the operation unit 1510, to display the spatial floating image 3.

[0289] Step S4 is the display of the spatial floating image 3 by the spatial floating image display device 1000. In step S4, the spatial floating image display device 1000 displays the spatial floating image 3 in its initial state. For example, an image 1520 of a wheel is displayed as in state A of FIG. 28 described later. The display in this initial state is a display that prompts user 230 to perform an air operation on the image 1520 of the wheel.

[0290] Step S5 is an air operation by user 230 on the image 1520 of the wheel in the spatial floating image 3, in other words, a product selection operation and a selection operation of the product selection button 1502. In step S6, the spatial floating image display device 1000 uses the user operation detection mechanism to detect the air operation in step S5 and generate detection information. The spatial floating image display device 1000 grasps, as information representing the air operation, for example, the displacement amount, displacement direction, etc. by this detection (FIG. 24 described later).

[0291] In step S7, the control unit 1110 of the spatial floating image display device 1000 transmits control information / signals for cooperation to the control unit 3501 corresponding to the detected air operation. This control information / signals may be information representing the air operation (such as the displacement amount), or product selection button selection information based on the mapping (FIG. 32) described later. Here, it is assumed that the control unit 1110 of the spatial floating image display device 1000 determines, by mapping, which product selection button 1502 has been selected from the information representing the air operation (such as the displacement amount), and generates product selection button selection information representing the selection state.

[0292] Also, in step S8, the video processing unit of the spatial floating video display device 1000 updates the display state of the wheel video 1520 in the spatial floating video 3 according to the air operation detected in step S6. This is, for example, a display where the wheel rotates according to the air operation.

[0293] On the other hand, in step S9, the control unit 3501 of the vending machine 3000 executes predetermined control based on the control information / signal received in step S7. Here, the predetermined control is to light the selected product selection button 1502 among the plurality of product selection buttons 1502 in the product posting unit 1501. As shown in FIG. 17 mentioned above, the control unit 3501 transmits a signal 1711 for the lighting control to the product selection button 1502. Based on the signal 1711, the light-emitting unit constituting the product selection button 1502 emits light.

[0294] The process / operation of selecting the product selection button 1502 corresponding to the above air operation (steps S5 to S9) can be repeated as many times as appropriate according to the intention of the user 230.

[0295] Step S10 is an operation for the user 230 to decide to purchase the selected product. This operation is an air operation of making a decision in the spatial floating video 3 (specifically, the decision button 2901 in FIG. 29 described later), or a contact operation of a physical decision button. In the case of an air operation (touch operation) of the decision button 2901 in the spatial floating video 3 as shown in FIG. 29 described later, the user 230 performs an air operation (touch operation) on the decision button 2901. When a physical decision button is provided outside the spatial floating video 3, the user 230 performs a contact operation (pressing) on the physical decision button.

[0296] When applying the determination button 2901 in the spatial floating image 3, in step S11, the spatial floating image display device 1000 uses the user operation detection mechanism to detect the air operation of the determination button 2901 as the purchase determination operation in step S10. When applying a physical determination button, the control unit 3501 of the vending machine 3000 detects the contact operation of the physical determination button. After step S11, the spatial floating image display device 1000 transmits control information / signal representing the purchase determination of the product to the control unit 3501 of the vending machine 3000.

[0297] In step S12, the control unit 3501 of the vending machine 3000 executes control corresponding to the determination of product purchase. In step S13, the user 230 performs operations of settlement and product acquisition. Correspondingly, in step S14, the control unit 3501 of the vending machine 3000 performs control of settlement and product vending.

[0298] [Air operation detection] FIG. 24 is an explanatory diagram of the air operation and detection by the finger 231 of the user 230 with respect to the wheel image 1520 in the spatial floating image 3 of the opening 1515 of the operation unit 1510. First, state A shows, as an example of the wheel image 1520, for clarity, one point on the spherical surface of the wheel is illustrated as point P, and the latitude and longitude lines passing through that point P are also illustrated. In the image 1520, such a point P and latitude and longitude lines may actually be displayed. Assume that the position coordinates of point P in the x-z plane of the display range 3R of the spatial floating image 3 are (x0, z0). For this wheel image 1520, the user 230 performs an air operation using the finger 231 (for example, one finger) as the operating object. Regarding the image 1520, the position where the finger 231 touches may be any position on the surface of the sphere. In state A, assume that the fingertip touches (contacts) point Q at a position slightly deviated from point P. Assume that the position coordinates of point Q in the x-z plane of the spatial floating image 3 are (x1, z1).

[0299] In the modification example, as long as the displacement amount or the like can be detected, the position where the finger 231 touches the x-z plane of the display range 3R of the floating image 3 may be an area outside the image 1520.

[0300] State B shows a case where, from State A, the user 230 moves the fingertip while touching (contacting) the x-z plane of the floating image 3, for example, roughly in the right direction (+x direction) from point Q. The position of the fingertip after the movement is point R. Let the position coordinates of point R in the x-z plane of the floating image 3 be (x2, z2). It is assumed that a certain time T is taken for this movement operation.

[0301] When the displacement amount (in other words, the position coordinate difference) during the movement of the touch point by this in-air operation is (Δx, Δz), then (Δx, Δz) = (x2 - x1, z2 - z1). Also, when the speed during this movement is (vx, vz), then (vx, vz) = (Δx / T, Δz / T). Further, this displacement amount has at least the x direction and the z direction as the displacement directions, and an oblique displacement direction as shown in the figure can also be grasped.

[0302] The floating image display device 1000 can detect, as a user operation detection mechanism, for example, the in-air operation on the wheel image 1520 as shown in FIG. 24, particularly the position coordinates, displacement amount, displacement direction, etc. of the touch point as detection results, using the in-air operation detection sensor 1351 and the in-air operation detection unit 1350 in FIG. 3, or the imaging unit 1180.

[0303] Point R in State B is a state where the fingertip is touching the x-z plane of the floating image 3, but it is not limited to this, and it may be a non-touch state such as a flick operation.

[0304] The spatial floating image display device 1000 can execute predetermined control based on the displacement amount, displacement direction, etc. as detection results by the above-mentioned air operation detection sensor 1351, etc. An example of the predetermined control is to select one product selection button 1502 of the product posting unit 1501 to a selected state according to the displacement amount and displacement direction and cooperate with the control unit 3501 (described later).

[0305] In addition, the video control unit controls to update the display of the wheel video 1520 based on the detection result of the above air operation. In other words, the display state is changed so that the wheel video 1502 reacts according to the air operation by the user 230. Thereby, the user 230 feels as if the wheel reacted and moved by his own air operation.

[0306] State C is an example after the display state is changed according to the detected air operation (displacement amount, etc.) from the display state of the wheel video 1520 such as states A and B. This display change is determined according to the detected displacement amount, etc. That is, for example, the larger the displacement amount and speed of the slide operation, the more the rotation state of the wheel video 1520 changes greatly in the display control. In this example, in the x-z plane of the spatial floating image 3, while the reference position of the wheel video 1520 is fixed, the spherical surface changes so as to rotate according to the displacement amount, etc. The point P in state A has moved to the point S (x3, z3) in state C.

[0307] The display change of the wheel video 1520 is not limited to the above example, and may be, for example, a change in color or brightness, or an effect display may be added. Also, not only the display but also voice output may be used in combination. For example, an operation sound may be output according to an operation such as a touch or slide of the wheel video 1520. The voice output can use the voice output unit 1140 in FIG. 3 or the voice output unit 1704 in FIG. 17.

[0308] Also, in the above example, the reference position of the wheel image 1520 was fixed. As a modification, within the x-z plane of the display range 3R of the floating-in-air image 3, the position of the wheel image 1520 may be varied according to an in-air operation. For example, when the user 230 performs a slide operation in the right direction, the wheel image 1520 may also be moved to a position slid in the right direction.

[0309] [Sensor of the operation unit: Camera] As a sensor for detecting an in-air operation on the wheel image 1520 of the floating-in-air image 3 in the operation unit 1510 of the product vending machine 3000, although the detailed configuration is not limited, for example, the following sensors are applicable.

[0310] As the sensor of the first example, a camera or a TOF sensor (TOF: Time Of Flight) of the imaging unit 1180 (FIG. 3) can be applied. FIG. 25 shows a side view of an arrangement example of the sensor 2501 when a camera or a TOF sensor is applied as the sensor of the first example. The sensor 2501 is arranged at a position where it can cover the imaging of the x-z plane of the floating-in-air image 3 from the back side. The sensor 2501 is arranged at a position on the back side (y direction) with respect to the floating-in-air image 3 such that the optical axis of imaging faces the front side (-y direction). In this example, the optical axis from the sensor 2501 is arranged to face obliquely downward. The x-z plane of the floating-in-air image 3 is included in the imaging range of the sensor 2501. Further, since the imaging range of the sensor 2501 also extends to the front side of the front surface 3591 via the opening 1515, the body of the user 230 in front of the floating-in-air image 3 can also be imaged. The floating-in-air image display device 1000 can determine the location touched by the finger 231 in the x-z plane of the floating-in-air image 3 based on analysis processing from the image of the camera which is the sensor 2501. Processing such as image analysis may be performed by the circuit of the imaging unit 1180 in FIG. 3, or may be performed by an image processing unit such as the in-air operation detection unit 1350, the control unit 1110, or the video control unit 1160.

[0311] When the sensor 2501 is a TOF sensor (in other words, a distance measurement sensor or the like), the light emitted from the TOF sensor is reflected at the location where the finger 231 touches in the x-z plane of the spatial floating image 3, and the TOF sensor receives the reflected light that returns. The TOF sensor can calculate the distance based on the time until the emitted light returns as reflected light. Therefore, based on that distance, it is possible to determine the location where the finger 231 touches in the x-z plane of the spatial floating image 3.

[0312] [Sensor of the operation unit: Air operation detection sensor] FIG. 26 shows, as a second example of the sensor applicable to the spatial floating image 3 of the operation unit 1510, a configuration example of the arrangement of the air operation detection sensor 1351 in a side view. The video processing unit such as the air operation detection unit 1350, the control unit 1110, or the video control unit 1160 in FIG. 3 uses the detection signal of the air operation detection sensor 1351 to detect and determine an air operation with respect to the display range 3R of the spatial floating image 3.

[0313] In FIG. 26, as an arrangement example of the air operation detection sensor 1351 in the opening 1515, on the front surface 3591 of the housing 3590, the air operation detection sensor 1351 is arranged downward at an upper position in accordance with the position in the depth direction (y direction) of the spatial floating image 3 of the opening 1515. The optical axis of the emitted light of the air operation detection sensor 1351 covers the x-z plane of the spatial floating image 3.

[0314] FIG. 27 shows, in an x-z plane view, a configuration example of the air operation detection sensor 1351 arranged above in accordance with the spatial floating image 3 of FIG. 26. In the air operation detection sensor 1351, a plurality of optical elements 1351c are arranged in the x direction. The optical element 1351c is a pair of a light emitting element 1351a and a light receiving element 1351b. The light emitting element 1351a is composed of, for example, an infrared element or the like. The light emitting element 1351a emits light a1, for example, infrared light, downward in the z direction. When the light a1 is not blocked by an object, it passes through the display range 3R. When the light a1 is blocked by an object, it is reflected by the object and returns as reflected light a2. The reflected light a2 is received by the light receiving element 1351b.

[0315] For example, when there is a contact point a3 by the user's finger in the x-z plane of the display range 3R, at that contact point a3, the light a1 is reflected and returns as the reflected light a2. The light receiving element 1351b at a certain position in the x direction detects the reflected light a2. Thereby, the air operation detection unit 1350 can know that there is a contact point a3 at that position in the x direction. Also, from the time until the light a1 returns as the reflected light a2, the distance can be calculated by the TOF method. For example, the distance a4 to the contact point a3 can be calculated. Thereby, the position coordinates of the contact point a3 in the x-z plane of the display range 3R can also be known.

[0316] Not limited to this example, the air operation detection sensor 1351 may be arranged above, below, or on the left or right with respect to the x-z plane of the display range 3R. Also, the air operation detection sensor 1351 may be arranged at a position shifted in the front-back direction, that is, the y direction, or a plurality of air operation detection sensors 1351 may be arranged at a plurality of positions in the front-back direction.

[0317] As in the above example, for the detection of an air operation (at least an operation in the x direction and the z direction) with respect to the image 1520 of the wheel in the x-z plane of the space floating image 3, it can be detected with sufficiently high accuracy by an air operation detection sensor 1351 of the type of a planar sensor as shown in FIG. 27, for example.

[0318] [Operation Unit of Vending Machine: Example of Air Operation and Display Control (1)] FIGS. 28 and 29 show an example of an air operation with respect to the image 1520 of the wheel of the operation unit 1510 and display control corresponding to the air operation. State A in FIG. 28 shows a state where the user 230 operates the image 1520 of the wheel by an air operation such as touching or sliding with the finger 231 with respect to the image 1520 of the wheel displayed in the space floating image 3 of the operation unit 1510. In the space floating image 3, not only the image 1520 of the wheel but also, for example, a guide image 2801 such as "Turn it to select a product" may be displayed, and at the same time, the guide content may be output as voice.

[0319] State B shows an example of the selection of product selection button 1502 in the product display section 1501 on the front surface 3591 of the housing 3590. In response to the operation in state A, as shown in state B, the selection state of the product selection button 1502 changes. In response to the air operation (for example, displacement in the x direction and the z direction) of the image 1520 of the wheel, the lighting position of the product selection button 1502 will be operated and controlled.

[0320] In this example, as an example of the products in the product display section 1501, 18 products from product A to product R are arranged in 3 rows and 6 columns as shown in the figure. And corresponding to those products, 18 product selection buttons 1502 are arranged in 3 rows and 6 columns. For the sake of identification, each product selection button 1502 may be described as product selection buttons A, B, C, etc.

[0321] In the example of state B, it is assumed that the initially selected product selection button 1502 (referred to as the first product selection button) is the product selection button 1502c corresponding to product C. And it is assumed that the product selection button 1502 (referred to as the second product selection button) selected by the user 230 through the air operation of the image 1520 of the wheel is the product selection button 1502q corresponding to product Q. The air operation on the image 1520 of the wheel at this time is a slide operation including a slide operation in the downward (-z direction) to a certain extent and a slide operation in the rightward (+x direction) to a certain extent. Note that this slide operation may be two sequential slide operations separated in the x direction and the z direction, or a slide operation in one diagonal direction (+x, -z). Depending on the displacement amount of the detection of the air operation as shown in FIG. 24, etc., the selected product selection button 1502 changes.

[0322] In response to a selection operation on the image 1520 of the wheel such as state A, like in state B, the product selection button 1502 is selected and controlled to light up the selected product selection button 1502. The selection state of the product selection button 1502 is updated at any time on the time axis. The control unit 3501 cooperates with the spatial floating image display device 1000 to control the product posting unit 1501 to light up the product selection button 1502 in the selected state. In the example of state B, it changes from the state where the first product selection button (C) is lit to the state where the second product selection button (Q) is lit.

[0323] Also, corresponding to this product selection operation, a change path 2802 of product selection as shown in the figure is configured. This path 2802 is a path connecting from the first product selection button (C) to the selected second product selection button (Q), and also has product selection buttons 1502 (I, O, P) passing through in the middle for connection. The control unit 3501 cooperates with the spatial floating image display device 1000 to at least light up the product selection button 1502 in the selected state (for example, product selection button 1502q), but not limited to this, and may also control the lighting of the product selection buttons 1502 in the path 2802. The control example will be described later.

[0324] Note that in this example, the case where the first product selection button initially selected is, for example, product selection button 1502c is shown, but it is not limited to this. In the initial state, none of the product selection buttons 1502 may be selected, or a product selection button 1502 at a predetermined default position may be selected (described later).

[0325] State C in FIG. 29 is a continuation from state B in FIG. 28. Similar to state B, when, for example, the product selection button Q (1502q) of product Q is selected, the spatial floating image display device 1000 switches and updates the display content of the spatial floating image 3 according to the selection. State C shows an example display after the switch. The spatial floating image display device 1000 displays the decision button 2901 as part of the images in the spatial floating image 3. In this example, the decision button 2901 is displayed at a position above the image 1520 of the wheel, but it is not limited to this and is possible otherwise. Also, not only the decision button 2901, but also a guide such as "Press the decision button when you have decided on the product" may be output. Also, in this example, the decision button 2901 is displayed, but it is not limited to this, and according to the flow up to the purchase of the product in the vending machine 3000, other display contents, for example, display of a purchase button, a payment button, etc. may be used.

[0326] Regarding the decision button 2901 in state C, if the user 230 continues or makes a product selection again without yet deciding on the product, the image 1520 of the wheel may be operated in the air again as in state A. In that case, the spatial floating image display device 1000 erases the decision button 2901 and performs control as in state A or state B.

[0327] In a modified example, the decision button 2901 may be constantly displayed from the beginning in the x-z plane of the spatial floating image 3. However, as will be described later, it is preferable to devise so that an accidental operation on the decision button 2901 is less likely to occur.

[0328] Regarding the decision button 2901 in state C, when the user 230 decides on the selected product, an air operation such as a touch operation is performed with the finger 231 to press the decision button 2901 as shown in state D. The spatial floating image display device 1000 detects the air operation and determines it as a purchase decision of the selected product.

[0329] State E is an example of the display in the spatial floating image 3 when transitioning to the phases of payment or product vending after state D. In this example, the image 1520 of the wheel has been erased, and a guide image 2902 such as "Please insert money" is being displayed, and at the same time, an audio output of the same content may also be made. In other examples, a guide such as "Please touch the payment card" regarding the payment device 1702 (Fig. 17) may also be used. When transitioning from a purchase decision to payment, the user 230 performs an operation on the coin / bill insertion unit 3593 or the payment device 1702, and the main body 3500 performs the payment process. After that, the main body 3500 vends the purchased product by the product vending mechanism 1708.

[0330] In the display control examples of FIGS. 28 and 29 above, initially, the decision button 2901 is not displayed, the image 1520 of the wheel is displayed, and an air operation on the image 1520 of the wheel is performed. After the selection state of the product selection button 1502 is generated by detecting the air operation, the decision button 2901 is displayed to accept the purchase decision operation. In other display control examples, during the execution of the air operation on the image 1520 of the wheel, that is, during detection, even if the selection state of the product selection button 1502 is generated, the decision button 2901 is not displayed. Then, after the generation of the selection state, a standby is made for a certain period of time or more, and when the state without an air operation continues for a certain period of time or more, the decision button 2901 may be displayed.

[0331] [Initial State] FIG. 30 shows, as an example of the initial state, an example of a state in which the product selection button 1502 at a predetermined default position regarding product selection is lit. In the example shown as the initial state A, the default position is the product selection button A (1502a) of product A in the upper left of the product posting section 1501. The control unit 3501 and the spatial floating image display device 1000 share the setting information (FIG. 32 described later) regarding the default position of the selection of the product selection button 1502. In the initial state A, the control unit 3501 controls to light the product selection button A (1502a).

[0332] In other examples, the default position may be the lower left, upper right, or lower right button among the plurality of product selection buttons 1502 of the product display section 1501, or may be any position on the upper, lower, left, or right side of the product display section 1501. In other examples, it may be the button (e.g., product selection button L) closest to the image 1520 of the wheel of the operation section 1510. When applying the initial state A, the default position is easy for the user 230 to understand. Note that the maximum value 3011 is an example of the maximum value of the movement amount when considering the movement from one product selection button 1502 to another product selection button 1502 within the product display section 1501.

[0333] In the example shown as the initial state B, the default position is a position near the center in the area of the product display section 1501. In this example, it is the product selection button I (1502i) of product I in the third column from the left in the second row. In this example, since there is no product selection button 1502 at the exact center position (the intersection of the dashed lines) among the plurality of product selection buttons 1502 of the product display section 1501, the product selection button I (1502i) adjacent to the left is set as the default position as the closest position to it. The product selection button J (1502j) adjacent to the right may be set as the default position. In the initial state B, the control unit 3501 controls to light up the product selection button I (1502i).

[0334] When applying the initial state B, considering the movement amount when the selected product selection button 1502 moves from the product selection button 1502 at the default position near the center to every other product selection button 1502, the maximum value of the movement amount can be minimized. The maximum value 3012 is the maximum value of that movement amount.

[0335] In the prior art, since there is no concept of product selection using the in-air operation of the operation unit 1510 or the image of the wheel 1520 by the spatial floating image 3, there is basically no concept such as the default position of the selection of the product selection button 1502 in the initial state as in the above example. In the third embodiment, since a concept / function of product selection using the in-air operation of the operation unit 1510 or the image of the wheel 1520 by the spatial floating image 3 is newly introduced, the default position of the selection of the product selection button 1502 in the initial state as in the above example can be utilized for control.

[0336] [Route of Changes in Product Selection] In the third embodiment, as in the example of state B in FIG. 28, control for expressing the route (in other words, the locus) at the time of change in product selection, that is, the route due to the change in the selection state of the product selection button 1502, by lighting the product selection button 1502 is possible. FIG. 31A shows an example of the lighting control of this route.

[0337] Example 1 of the route control is the same as state B in FIG. 28. The first product selection button initially selected is the product selection button C (1502c) for product C, and the second product selection button subsequently selected by the in-air operation of the image of the wheel 1520 is the product selection button Q (1502q) for product Q. The control unit 3501 changes from the state where the first product selection button (C) is lit to the state where the second product selection button (Q) is lit. In other words, at the first time point, only the first product selection button (C) is lit, and at the subsequent second time point, only the second product selection button (Q) is lit. In this example, on the route 3101 configured according to the in-air operation, the product selection buttons 1502 passing through in the middle are controlled not to be lit. The user 230 can recognize that the selected product (the corresponding product selection button 1502) has changed from C to Q by seeing such lighting.

[0338] As for Example 2 of path control, the difference from Example 1 is that on the same path 3101, the product selection buttons 1502 (I, O, P) passed through on the way are also controlled to light up sequentially. That is, starting from the state where the product selection button C is lit first, in response to an air operation, the product selection buttons I, O, P light up in order, and finally, the selected product selection button Q lights up. The product selection buttons 1502 (I, O, P) passed through on the way are in a temporarily selected state in a short time in response to an air operation. By looking at the lighting on such a path 3101, the user 230 can more clearly recognize that the selected product (the corresponding product selection button 1502) has changed from C to Q.

[0339] Also, as a modification regarding Example 2, the following may be used. In the processing of the spatial floating image display device 1000, in response to an air operation, only two product selection buttons 1502 in the selected state are the first product selection button (C) and the second product selection button (Q), and it is assumed that the product selection buttons I, O, P are not in the selected state. In this case, the control unit 3501 configures the same path 3101 from the product selection button C to the product selection button Q, and on this path 3101, the product selection buttons I, O, P passed through on the way are lit sequentially starting from the product selection button C, and finally the product selection button Q is lit. In this case, the product selection buttons I, O, P passed through on the way are not even temporarily selected, but are controlled to light up in order to show the path 3101 to the user 230 more clearly.

[0340] Although not shown in the figure, in other examples, it is possible to represent a path diagonal to the x direction and the z direction. For example, a diagonal path from the product selection button C to the product selection button Q can be configured. The path is, for example, a path that moves sequentially to the product selection buttons C, J, Q.

[0341] Example 3 of path control is the case where the default position of product selection in the initial state is product A (product selection button A) (initial state A in Fig. 30). Path 3103 is a path connecting from product selection button A to product selection button Q selected by an in-air operation. Also, in this example, the product selection button A at the default position is controlled not to light up.

[0342] Example 4 of path control is the case where, as a difference from Example 3, the product selection button A at the default position in the initial state is also controlled to light up. When starting from the default position as in Example 3 and Example 4, similar to Example 2, the control to light up the product selection button 1502 passing through on the path can also be similarly applied.

[0343] Fig. 31B shows, as a modification example, another example of path control. State A is an example where user 230 inserts coins / bills as described above, and all product selection buttons 1502 of products that can be purchased within the inserted amount range are in a predetermined lighting state (first lighting). In this example, 12 products A to L in the first and second rows are products that can be purchased, and the corresponding product selection buttons 1502 (A to L) are in the state of the first lighting. Here, the first lighting is indicated by an ellipse.

[0344] State B shows the case where user 230 selects the product selection button 1502k of one product (for example, product K) from the product selection buttons 1502 (A to L) in the state of the first lighting in state A by an in-air operation of the wheel video 1520. The selected product selection button 1502k is in a state of a predetermined lighting (second lighting). Here, the second lighting is indicated by a star. In this case, similar to the path control example in Fig. 31A, the vending machine 3000 controls the lighting (second lighting) of the product selection button 1502 on the path from the product selection button 1502 before selection (in other words, before movement) to the product selection button 1502 after selection (in other words, after movement). For example, path 3105 from product A to product selection button 1502k of product K is configured. This path 3105 is configured within the range of the first lighting.

[0345] [Cooperation between the control unit of a vending machine and a spatial floating image display device] FIG. 32 shows a configuration example regarding the cooperation between the control unit 3501 of the vending machine 3000 and the spatial floating image display device 1000. In particular, in this figure, a configuration example regarding the control of the mapping (association) between the aerial operation of the image 1520 of the wheel and the selection of the product selection button 1502 is shown. The product display unit 1501 has a system for arranging a plurality of product selection buttons 1502 according to the implementation. For control and cooperation, the control unit 3501 and the spatial floating image display device 1000 grasp this system. At least one of the control unit 3501 and the spatial floating image display device 1000 performs the mapping (association) between the aerial operation and the selection of the product selection button 1502 based on this system.

[0346] The spatial floating image display device 1000 includes an image processing unit 3201, a display unit 3202, an optical system 3203, a user operation detection mechanism 3204, etc. The image processing unit 3201 can be constituted by the control unit 1110, the image control unit 1160, or the aerial operation detection unit 1350, etc. in FIG. 3. The display unit 3202 corresponds to the display device 1, etc. in FIG. 3. The optical system 3203 corresponds to the retroreflective unit 1101, etc. in FIG. 3. The user operation detection mechanism 3204 corresponds to the imaging unit 1180 and the aerial operation detection sensor 1351 in FIG. 3.

[0347] Based on the video processing of video information / video data, the image processing unit 3201 creates data / signals for displaying an image on the screen of the display unit 3202 and drives and controls the display unit 3202. The display unit 3202 displays an image on the screen (for example, the screen of the liquid crystal display panel 11 described above) according to the data / signals. The image light emitted corresponding to the image on the screen of the display unit 3202 is adjusted through the optical system 3203 to form a spatial floating image 3, which is a real image, at a predetermined position. The user 230 performs an aerial operation as a user operation on the screen by the spatial floating image 3. The user operation detection mechanism 3204 detects the aerial operation on the screen by the spatial floating image 3. The image processing unit 3201 executes a predetermined process based on the detection information by the user operation detection mechanism 3204.

[0348] In FIG. 32, the video processing unit 3201 cooperates with the control unit 3501 of the vending machine 3000 through communication via the communication unit 1132 in FIG. 3 and the communication interface 3505 in FIG. 16. Based on the information from the video processing unit 3201, the control unit 3501 controls the lighting of the product selection button 1502 on the product display unit 1501. Note that on the side of the vending machine main body 3500, when the product selection button 1502 is pressed by a contact operation, etc., the control unit 3501 may cooperate with the video processing unit 3201 via communication for predetermined control.

[0349] In FIG. 32, at least one of the spatial floating video display device 1000 (particularly the video processing unit 3201) and the vending machine main body 3500 (particularly the control unit 3501) has a function of performing mapping regarding the in-air operation of the wheel video 1520 and the selection state of the product selection button 502. In this embodiment, it is assumed that the video processing unit 3201 has a function 3211 for performing such mapping. However, in a modified example, the control unit 3501 may have a function 3212 for performing such mapping.

[0350] When the video processing unit 3201 has a function 3211 for performing mapping, the following is the case. In response to detecting an in-air operation on the wheel video 1520 in the spatial floating video 3, the video processing unit 3201 selects the product selection button 1502 by means of mapping processing. Specifically, the video processing unit 3201 manages and holds in the product selection management information 3221, that is, the mapping information, the configuration information of the system of the plurality of product selection buttons 1502 on the product display unit 1501 and the information indicating which product selection button 1502 is currently in a selected state. The video processing unit 3201 can select the product selection button 1502 from this product selection management information 3221 and the detection information such as the displacement amount (FIG. 24). Then, the video processing unit 3201 transmits the product selection button selection information 3231 to the control unit 3501 as information representing the selected product selection button 1502. Based on the received product selection button selection information 3231, the control unit 3501 controls to light up the selected product selection button 1502.

[0351] When the control unit 3501 has a function 3212 for performing mapping, the following applies. When the video processing unit 3201 detects an air operation on the wheel video 1520 in the spatial floating video 3, the video processing unit 3201 transmits detection information 3232 such as the aforementioned displacement amount to the control unit 3501. Based on the received detection information 3232, the control unit 3501 selects the product selection button 1502 by mapping processing. Specifically, the control unit 3501 manages and holds in the product selection management information 3222, that is, the mapping information, the configuration information of the system of the plurality of product selection buttons 1502 of the product posting unit 1501 and the information indicating which product selection button 1502 is currently in the selected state. The control unit 3501 can select the product selection button 1502 from the product selection management information 3222 and the detection information 3232. Then, the control unit 3501 controls to light up the selected product selection button 1502.

[0352] Also, the control unit 3501 may appropriately transmit to the video processing unit 3201 the information indicating which product selection button 1502 is currently in the selected state.

[0353] Note that the following may be included as the setting information regarding the above mapping. The setting information includes a setting value that determines, when the displacement amount of the air operation is at a certain level, whether to move the selection of the product selection button 1502 on the product posting unit 1501 one unit up, down, left, or right (in other words, move to the adjacent button).

[0354] [Prevention of accidental operation of the decision button (1)] FIG. 33 shows a configuration example of the operation unit 1510 in a modified example of the third embodiment in an x-z plane view looking at the front surface 3591 of the housing 3590. The purchase decision operation / processing corresponding to the pressing of the aforementioned decision button 2901 is important, and it is desirable to prevent accidental operation. Therefore, in this modified example, in order to prevent accidental operation, a partition 3301, which is a physical frame, is provided near the decision button 2901. This partition 3301 partitions the space so as to protect the decision button 2901 and is an object that can stop the finger 231.

[0355] In this example, at the opening 1515 where the floating image 3 is arranged, a partition 3301 is provided corresponding to the position where the above-described determination button 2901 is displayed. This partition 3301 is provided in a shape surrounding the periphery, top, bottom, left, and right of the determination button 2901 in the x-z plane. This partition 3301 is physically connected to the end of the opening 1515 which is a part of the housing 3590. Note that the partition 3301 is not limited to this shape and can be provided at least at a position between the image 1520 of the wheel and the image of the determination button 2901.

[0356] A side view is also shown on the lower side. As shown in the figure, the partition 3301 has a width in the depth direction (y direction). The floating image 3 is arranged within this width range. The lower side portion of the partition 3301 is arranged at a position between the image of the determination button 2901 and the image 1520 of the wheel. When the user 230 performs an in-air operation such as sliding on the image 1520 of the wheel with the finger 231 and the finger 231 accidentally slips and moves towards the determination button 2901, as shown in the figure, the finger 231 hits the partition 3301 and is stopped. Therefore, an erroneous operation of accidentally touching (pressing) the determination button 2901 can be prevented. When the user 230 intentionally touches (presses) the determination button 2901, the finger 231 is once moved back a little to the front side (-y direction) avoiding the partition 3301, and then, as shown in the figure, the finger 231 is made to enter the space inside the partition 3301 on the back side again, so that the determination button 2901 can be touched (pressed).

[0357] In this example, since the determination button 2901 is composed of the floating image 3, it is also easy to change it to the display of other object images. In this example, the case of the determination button 2901 is shown as an example of an important object image for which erroneous operations are to be prevented, but it is not limited to this, and the same configuration such as the partition 3301 can be applied to other important object images as well.

[0358] As a general characteristic, when a user views a spatially floating image from their perspective, it may be difficult to perceive a sense of distance or the like regarding the recognition of an object image in the depth direction. For this reason, there is also a possibility that the user may accidentally operate incorrectly due to a misjudgment of the sense of distance during an aerial operation on the object image. Therefore, in order to prevent incorrect operations, a partition 3301, which is a physical frame, may be provided. From the user's perspective, the partition 3301 allows for a clearer perception of the sense of distance in the depth direction. In relation to this partition 3301 and the object image, incorrect operations on the object image (for example, the determination button 2901) can be prevented.

[0359] [Prevention of Incorrect Operation of the Determination Button (2)] FIG. 34 shows a configuration example of the operation unit 1510 in a modification regarding the partition 3301 of FIG. 33. On the front surface 3591 of the housing 3590, in the opening 1515, a frame 3401 (corresponding to the frame 1530 in FIG. 20B), which is a physical frame, is provided so as to surround the entire display range 3R of the spatially floating image 3 in all directions of up, down, left, and right. Furthermore, a partition 3402 is provided so as to partition the spatial region within this frame 3401. This partition 3402 divides the region of the image 1520 of the wheel and the region of the image of the determination button 2901. The frame 3401 and the partition 3402 also have a width in the front-back direction in the depth direction with respect to the position of the front surface 3591. In the configuration example of FIG. 34 as well, similar to FIG. 33, incorrect operations can be prevented.

[0360] FIG. 35 shows a modification of the partition 3301 in FIG. 33. In this modification, on the front surface 3591 of the housing 3590, there are two separated openings 1515, namely, opening 1515-1 and opening 1515-2. Inside one of the openings 1515-1, an image 1520 of a wheel is displayed as part of the spatial floating image 3, and inside the other opening 1515-2 which is separated above, an image of the determination button 2901 is displayed as part of the spatial floating image 3. Also, a frame having a width in the depth direction as described above may be provided for each opening 1515. In this modification, it can be said that the housing portion 3403 between the two openings 1515 functions in the same manner as the partition 3301 or the partition 3402 described above.

[0361] [Prevention of accidental operation of the determination button (3)] FIG. 36A shows a configuration example of the operation unit 1510 in a modification of Example 3. In FIG. 36A, on the front surface 3591 of the housing 3590, a physical determination button 3601 is provided outside the opening 1515, for example, at an upper position. When the user 230 selects a product by operating the image 1520 of the wheel in the opening 1515 with a finger 231 in the air and then decides to purchase the product, the finger 231 is moved outside the opening 1515 and the determination button 3601 is pressed as a contact operation. When the user 230 is operating the image 1520 of the wheel by sliding or the like, if the finger 231 is moved toward the upper determination button 3601, it hits the upper side portion of the opening 1515 and is stopped. Therefore, accidental operation of the determination button 3601 is prevented.

[0362] As another modification, as shown in FIG. 36B, a physical cancel button 3602 may be provided outside the opening 1515. In this case, when the user 230 wants to cancel the product selection, the cancel button 3602 is pressed as a contact operation with the finger 231. In response to this pressing, the vending machine 3000 cancels the product selection state, in other words, resets it and returns to, for example, the initial state. The user 230 can also start over.

[0363] [Prevention of accidental operation of the determination button (4)] FIG. 37A shows a configuration example of the operation unit 1510 in a modification (referred to as the first example) of Example 3 in a side view. In this first example, although the concept is the same as that in FIG. 20B and the like, on the front surface 3591 of the housing 3590, at a position protruding forward by a distance d1, an image 1520 of a wheel and an image of the determination button 2901 are formed as the spatial floating image 3. Note that the protruding distance of the spatial floating image 3 can be appropriately designed according to the various configuration examples described above. Also, in this example, in order to prevent an accidental operation of the determination button 2901, a partition 3701, which is a physical frame, is provided in the same manner as described above (FIG. 33 and the like). This partition 3701 is located at a position overlapping the spatial floating image 3 in the depth direction (y direction), that is, at the position of the distance d1. This partition 3701 has a width d2 in the y direction.

[0364] Also, in FIG. 37A, in order to dispose this partition 3701 at the illustrated position in the space, a housing connection portion 3702 for connecting this partition 3701 and the front surface 3591 of the housing 3590 is also provided. In this example, the partition 3701 is connected to an upper position of the opening 1515 in the front surface 3591 of the housing 3590 via the housing connection portion 3702. One side of the housing connection portion 3702 is fixed to the front surface 3591, and the other side is fixed to the partition 3701. The housing connection portion 3702 can apply, for example, a metal plate and screwing, etc., but is not limited thereto.

[0365] The image 1520 of the wheel is arranged to float at a position below the partition 3701. When the user 230 operates the image 1520 of the wheel with the finger 231 and accidentally moves the finger 231 toward the determination button 2901, the finger 231 is blocked by the partition 3701 (particularly the lower end portion of the partition 3701), and the moving range is restricted. Therefore, an accidental operation of the determination button 2901 can be prevented.

[0366] Also, FIG. 37A shows an example of the arrangement of the sensor 3710 for detecting an in-air operation. In this example, the sensor 3710, such as the aforementioned camera or TOF sensor, is arranged in consideration of the fact that it can detect the image 1520 of the wheel with high precision. The optical axis of the sensor 3710 is arranged to face the center position of the image 1520 of the wheel, and the detection range of the sensor 3701 covers the image 1520 of the wheel.

[0367] Also, in this example, a sensor 3720 different from the sensor 3710, such as an infrared sensor, is provided on the partition 3701. This sensor 3720 only needs to detect the in-air operation (touch operation) of the determination button 2901, and does not require detection of displacement or the like like the sensor 3710. For example, when the infrared rays from the infrared sensor, which is the sensor 3720, are blocked by the finger 231, it is detected as the pressing of the determination button 2901. It is also possible to apply the in-air operation detection sensor 1351 as the sensor 3720.

[0368] As in this example, the sensor for detecting the in-air operation of the image 1520 of the wheel and the sensor for detecting the operation of the determination button 2901 may be configured separately as different sensors and detection mechanisms.

[0369] Regarding the structure of the partition 3701 and the positional relationship and structural details in the depth direction (y direction) with the spatial floating image 3, a plurality of configuration examples are as follows, including FIG. 37A. Any of them is applicable.

[0370] FIG. 37B shows a second example. In the second example, the spatial floating image 3 is formed at a position closer to the front with respect to the position in the depth direction (y direction) of the partition 3701. In other words, the partition 3701 is arranged at a position deeper than the position where the spatial floating image 3 is formed. The distance d3 is the distance from the front surface 3591 to the spatial floating image 3. The distance d4 is the distance from the front surface 3591 to the partition 3701. The distance d3 is larger (d4 < d3). The image light that forms the image of the determination button 2901 emitted from the optical system of the spatial floating image display device 1000 forms the image of the determination button 2901 at the illustrated position via the space inside the partition 3701 in the y direction. Specifically, as shown in the figure, since a part of the image light A1 is blocked by the partition 3701 (particularly the lower end of the partition 3701), the formed spatial floating image 3 is divided into two parts. In the second example, since the spatial floating image 3 is formed in front of the partition 3701, for example, as shown in the figure, if the air operation detection sensor 1351 is provided at the housing connection portion 3702, the operation detection for the spatial floating image 3 can be performed by one air operation detection sensor 1351 without being blocked by the partition 3701. That is, with one air operation detection sensor 1351, it is possible to detect operations for both the operation of the determination button 2901 and the image 1520 of the wheel.

[0371] In the second example, as seen from the user 230, the object image of the spatial floating image 3 appears closer to the front than the partition 3701. In other words, it does not seem that the partition 3701 appears in front of the spatial floating image 3. That is, as seen from the user 230, a stronger sense of floating can be felt for the decision button 2901. In the second example, there is an advantage that no separation of the object image by the partition 3701, in other words, no shielding of the video light, etc. occurs between the viewpoint of the user 230 and the object images such as the decision button 2901. Also, in the second example, the presence of the partition 3701 arranged immediately behind the object image can prevent an accidental operation of the decision button 2901. That is, when the user 230 touches and operates the object image with the finger 231, ultimately, the finger 231 interferes with the partition 3701 (especially the lower end of the partition 3701) at the point where the finger 231 penetrates the spatial floating image 3 to the back side, and prevents the finger 231 from moving to the object image in the internal space of the partition 3701. Therefore, an accidental operation on, for example, the decision button 2901 in the internal space of the partition 3701 can be prevented.

[0372] Figure 37C shows a third example. In the third example, the floating image 3 is formed at a position deeper in the depth direction (y direction) of the partition 3701, i.e., the partition 3701 is arranged at a position closer to the front side with respect to the position where the floating image 3 is formed. The distance d5 is the distance from the front surface 3591 to the floating image 3, and the distance d6 is the distance from the front surface 3591 to the partition 3701. The distance d6 is greater (d5 < d6). Also, in the third example, the width d7 of the partition 3701 in the depth direction is smaller than that in the first example and the like. In the third example, the determination button 2901 is visible at a position slightly deeper than the partition 3701 as viewed from the user 230. In the third example, the partition 3701 appears closer to the front side, and the erroneous operation of the determination button 2901 can be prevented by the partition 3701. In the third example, since the floating image 3 is formed deeper than the partition 3701, for example, as shown in the figure, if the air operation detection sensor 1351 is provided at the housing connection portion 3702, the operation detection for the floating image 3 can be performed by one air operation detection sensor 1351 without being blocked by the partition 3701. That is, the operation detection for both the operation of the determination button 2901 and the image 1520 of the wheel can be performed by one air operation detection sensor 1351. Also, in this arrangement, since the air operation detection sensor 1351 can be arranged on the back side of the housing connection portion 3702 as viewed from the user, an excellent design can be realized without making the user feel the presence of the air operation detection sensor 1351.

[0373] In the third example, since the partition 3701 is located closer to the front side with respect to the object image of the floating image 3, when the user 230 operates the image 1520 of the wheel with the finger 231 and accidentally moves the finger 231 toward the determination button 2901, the finger 231 is blocked by the partition 3701 (especially the lower end portion of the partition 3701). Therefore, the erroneous operation of the determination button 2901 can be more reliably prevented.

[0374] The configuration examples regarding the positional relationship in the depth direction described above can be similarly applied to the various configuration examples (Figs. 20A to 20D, etc.) described above.

[0375] [Modification Example: Operation in the Depth Direction] In the above-described Example 3, the in-air operation and detection with respect to the wheel image 1520 are only the touch (contact) of a finger with respect to the x-z plane of the spatially floating image 3 and the direction within the x-z plane, and the in-air operation and detection in the depth direction (y direction) are not used. In a modification example, the in-air operation and detection in the depth direction (y direction) may be further used. When this is used, the spatially floating image display device 1000 is provided with a sensor capable of detecting an in-air operation in the depth direction (y direction) as a user operation detection mechanism. As an example of the sensor, an imaging unit 1180 such as a camera arranged so as to be able to photograph the movement of a finger in the depth direction may be used, or a plurality of in-air operation detection sensors 1351 arranged in the depth direction may be used.

[0376] In FIG. 38, an example of a pushing operation as an in-air operation in the depth direction (y direction) in the modification example and an example of a sensor capable of detecting the pushing operation are shown in a side view. For example, after the finger 231 of the user 230 touches the wheel image 1520, it penetrates in the depth direction, which is the back side, in the y direction. The distance of penetration from the position of the spatially floating image 3 to the back side is indicated by the distance d38. In the space within the opening 1515, the state in which the finger 231 is penetrating is maintained for a certain period of time or more. When the spatially floating image display device 1000 detects such an operation of the finger 231, it detects and determines that it is a pushing operation in the depth direction.

[0377] In FIG. 38, as an example of a sensor capable of detecting a pushing operation, a plurality of in-air operation detection sensors 1351 are arranged above the spatially floating image 3. A plurality of in-air operation detection sensors 1351, for example, three in-air operation detection sensors 1351 arranged in front and back with the position of the spatially floating image 3 as the center, are each arranged downward. With this sensor, the position of the finger 231 (for example, the fingertip) in the depth direction (y direction) can be detected. Also, not limited to this, even when a camera is installed downward at the same position, similarly, the position of the finger 231 (for example, the fingertip) in the depth direction (y direction) can be detected.

[0378] In addition, the spatial floating image display device 1000 and the vending machine 3000 may perform predetermined control in response to detection of an air operation in the depth direction (y direction) such as the example of the pushing operation described above. Examples of this predetermined control are as follows.

[0379] First example: Assume that the touch and slide operations (operations in the x and z directions) on the wheel image 1520 are the selection of the product selection button 1502 as described above. When the user 230 decides on the product selection, it may be configured to operate a dedicated decision button (the decision button in the spatial floating image 3 or the decision button in the hardware). In a modified example, instead of using a dedicated decision button, an operation in the depth direction on the wheel image 1520, for example, a pushing operation, is received, and this pushing operation is received as a "decision" operation. For example, when the spatial floating image display device 1000 detects this pushing operation, it executes a purchase decision process for the product selection. That is, when this "decision" operation is performed while a certain product selection button 1502 is selected, it will transition to the purchase of the product corresponding to the selected product selection button 1502. In this modified example, the user 230 can make a purchase decision only by the air operation of the wheel image 1520 without operating another dedicated decision button or the like.

[0380] Second Example: For the image 1520 of the wheel, assume that touch and slide operations (operations in the x - direction and z - direction) are the selection of the product selection button 1502 as described above. When user 230 cancels (or in other words, resets) the product selection, it may be configured to operate a dedicated button (the cancel button in the floating - in - space image 3 or the cancel button on the hardware). In a modified example, instead of using a dedicated cancel button, an operation in the depth direction with respect to the image 1520 of the wheel, for example, a push - in operation, is received, and this push - in operation is received as a "cancel" operation or a "reset" operation. For example, when the floating - in - space image display device 1000 detects this push - in operation, it executes the cancel process of the product selection and returns to the initial state regarding the product selection. The initial state is a state where none of the product selection buttons 1502 are selected, or a state where the product selection button 1502 at a predetermined default position is selected. In this modified example, user 230 can cancel the product selection only by the in - air operation of the image 1520 of the wheel without operating another dedicated cancel button or the like.

[0381] [Modified Example: Product Image Display] As a modification of the above-described Example 3, the following is also possible. FIG. 39 shows the operation unit 1510 in this modification. In this modification, in the spatially floating image 3 of the operation unit 1510, together with the image 1520 of the wheel, an image 3901 of a product associated with a product selection button 1502 that becomes a selected state in response to an aerial operation of the image 1520 is also displayed. In this example, in the display range 3R of the spatially floating image 3, the image 1520 of the wheel is displayed at the lower part, and the image 3901 of the product is displayed at the upper part. The action by the aerial operation on the image 1520 of the wheel is the same as described above. In response to a certain aerial operation, for example, a product selection button 1502X of a certain product X (for example, a beverage) is selected, and the product selection button 1502X lights up. Along with that, an image 3901 of the product X associated with the product selection button 1502X is displayed. The image 3901 may have information such as, for example, a product image, a product number, a product name, a product description, a price, an advertisement, etc. When the user 230 continues the aerial operation of the image 1520 of the wheel, the display content of the image 3901 of the product is switched according to the change in the selected state thereby. The user 230 can determine the desired product to purchase while looking not only at the product posting section 1501 but also at this image 3901. After the product to be purchased is determined, the state in which the image 3901 of the product is displayed corresponds to the state in which the product selection button 1502 corresponding to the product is selected. Therefore, thereafter, the user 230 may proceed with the operation of the determination button 2901 and the settlement procedure in the same manner as described above.

[0382] In this modification, it can be said that the image 3901 of the product, which is a part of the spatially floating image 3, also functions as the product posting section 1501. In the case of this modification, when the user 230 performs an aerial operation on the image 1520 of the wheel, it is also possible to omit the step of looking at and checking the products in the product posting section 1501.

[0383] Also, in the example of FIG. 39, an image representing a product number or the like of the product selected by the aerial operation of the image 1520 of the wheel may be displayed in a part of the spatially floating image 3 or in a display section separate from the spatially floating image 3.

[0384] <Example 4> As Example 4 of the present invention, a configuration example of a spatial floating image display device will be described. As a basic configuration, Example 4 is the same as Example 3. In this Example 4, the differences from Example 3 will be described, and the description of the same configuration as in Example 3 will be omitted repeatedly.

[0385] The spatial floating image display device of Example 4 is applied to an automatic ticket vending machine installed in a public facility such as a station or a store, etc., and is provided as one component in the automatic ticket vending machine. This spatial floating image display device is provided as a UI for operations / inputs such as selection of items / buttons from a menu screen for ticket purchase, etc. In other words, this spatial floating image display device functions as an operation unit, an input unit, a display unit, and a UI unit of the automatic ticket vending machine.

[0386] In Example 4 shown in FIG. 40 etc., similar to Example 3, a video 4020 of a wheel is displayed on the operation unit 4010 of the automatic ticket vending machine 4000, and in response to an in-air operation of the user 230 on the video 4020 of the wheel, a selection operation and determination of an arbitrary object video (for example, a menu item or a button) displayed within the screen of the display 4001 are enabled. Also, in Example 4, similar to Example 3, a determination button etc. may be displayed within the spatial floating image 3 of the operation unit 4010. Also, in Example 4, similar to Example 3, for an object video (for example, a determination button) for which it is desired to prevent misoperation, the periphery may be configured to be surrounded by a partition which is a physical frame. Also, in Example 4, similar to Example 3, the video 4020 of the wheel and the video of the determination button may be separated by an opening.

[0387] The automatic ticket vending machine 4000 may be a device such as a change machine, a reception machine, etc. Not limited to the automatic ticket vending machine 4000, it is similarly applicable to a predetermined device that requires a selection operation of an item / button on the screen of a display.

[0388] [Automatic Ticket Vending Machine] FIG. 40 shows a configuration example of a ticket vending machine 4000 including a spatial floating image display device 1000 of Example 4 as an operation unit 4010. The operation unit 4010 may include any one of the spatial floating image display devices 1000 described in Example 1 or Example 2. Hereinafter, in various descriptions of the ticket vending machine of this embodiment, the spatial floating image display device 1000 that is the operation unit 4010 has, for example, the configuration shown in FIG. 3. The operations of the respective components of the spatial floating image display device 1000 that is the operation unit 4010 are controlled by the control unit 1110 as described in FIG. 3. The video display operation of the spatial floating image display device 1000 that is the operation unit 4010 is displayed on the display device 1 based on the control of the video control unit 1160 as described in FIG. 3. The operation detection operation of the spatial floating image display device 1000 that is the operation unit 4010 is such that, as described in FIG. 3, based on the sensing result by the air operation detection sensor 1351, the air operation detection unit 1350 performs an operation detection process and generates control information based on the operation detection. Since the functions and operations of the other configurations shown in FIG. 3 are as described in Example 1 or Example 2, repeated descriptions are omitted. FIG. 40 shows a perspective view when viewing the front surface 4091 of the housing 4090 of the ticket vending machine 4000 from the user 230. A display 4001 (for example, a liquid crystal display panel) having a two-dimensional screen is provided on the upper front surface 4091 of the housing 4090 of the ticket vending machine 4000. The upper front surface 4091 and the screen of the display 4001 are slightly inclined so as to be easily visible from the user 230.

[0389] The ticket vending machine 4000 in FIG. 40 includes a display 4001 (for example, a liquid crystal display panel) having a two-dimensional screen on the upper front surface 4091 of a housing 4090 standing in the vertical direction. The upper front surface 4091 and the screen of the display 4001 are slightly inclined so as to be easily visible from the user 230.

[0390] Also, on the front surface 4091, an operation unit 4010 by the spatial floating image display device 1000 is provided at a position below the display 4001. In the operation unit 4010, a spatial floating image 3 is formed in the opening 4015, and an image 4020 of a wheel is displayed by the spatial floating image 3. The image 4020 of the wheel is the same as the image 1520 in the third embodiment. The user 230 performs an air operation on the image 4020 of the wheel with the finger 231.

[0391] [Configuration Example of Ticket Vending Machine] As a configuration example, the ticket vending machine 4000 in FIG. 40 can similarly apply the configuration example in FIG. 16. The commodity vending machine main body 3500 in FIG. 16 is replaced by the ticket vending machine main body, and the commodity vending machine functional unit 3510 is replaced by the ticket vending machine functional unit. In the example of the ticket vending machine 4000 as well, the spatial floating image display device 1000 which is the operation unit 4010 communicates with the communication interface of the ticket vending machine main body via the communication unit 1132 under the control of the control unit 1110 in FIG. 3. Through this communication, various control information such as control information based on the detection of air operation can be transmitted from the spatial floating image display device 1000 to the ticket vending machine main body. In addition, the external power input interface 1111 in FIG. 3 of the spatial floating image display device 1000 which is the operation unit 4010 is connected to the power supply of the ticket vending machine main body. Thereby, a power supply for operating the spatial floating image display device 1000 which is the operation unit 4010 is ensured.

[0392] FIG. 41 shows a configuration example of the ticket vending machine functional unit 4510 of the ticket vending machine 4000. Components such as the display 4001 in FIG. 41 are connected to the control unit 4501 of the ticket vending machine 4000. The ticket vending machine functional unit 4510 is a component for realizing the functions of the ticket vending...

Claims

1. An airborne floating image display device mounted on a vending machine or a ticket vending machine, wherein the airborne floating image display device comprises an image processing unit that performs image processing, a display unit that displays the image processed by the image processing unit, an optical system that generates an airborne floating image based on the image displayed by the display unit, a user operation detection mechanism that detects an operation by a user on the display range of the airborne floating image, a control unit, a communication interface, and is equipped with them, the airborne floating image display device constitutes an operation unit for operating the vending machine or the ticket vending machine, the airborne floating image display device displays the airborne floating image for operating the vending machine or the ticket vending machine, the control unit of the airborne floating image display device outputs, via the communication of the communication interface, information for executing a predetermined process by the vending machine or the ticket vending machine based on the detection of an operation on the airborne floating image, the predetermined process includes a process of selecting one from a plurality of candidates, the airborne floating image includes an object image that accepts an operation for the selection, Airborne floating image display device.

2. In the airborne floating image display device according to Claim 1, the airborne floating image display device displays, in the airborne floating image, an image that accepts an operation in the direction within the plane of the airborne floating image, Airborne floating image display device.

3. In the airborne floating image display device according to Claim 2, the image is an image of a wheel or a sphere, Airborne floating image display device.

4. In the airborne floating image display device according to Claim 1, the control unit of the airborne floating image display device performs control to output, via the communication of the communication interface, information for determining the selection in the predetermined process based on the detection information of the operation on the airborne floating image, Airborne floating image display device.

5. In the airborne floating image display device according to Claim 1, the predetermined process includes a process related to the selection and purchase of products by the vending machine, or a process related to the selection and ticket issuance of the ticket vending machine, Airborne floating image display device.

6. In the airborne floating image display device according to Claim 1, the predetermined process includes a process related to the selection and purchase of products by the vending machine, The predetermined process includes a process of selecting one product selection button from a plurality of product selection buttons on a product display section or a display of the product vending machine. Aerial floating image display device.

7. In the aerial floating image display device according to claim 6, The control unit of the aerial floating image display device controls to output, via communication of the communication interface, information for lighting the selected product selection button in the product vending machine. Aerial floating image display device.

8. In the aerial floating image display device according to claim 7, The information for lighting the selected product selection button is information for controlling to light the product selection button in a path from a first product selection button to a second product selection button selected when lighting the selected product selection button in the product vending machine. Aerial floating image display device.

9. In the aerial floating image display device according to claim 1, The operation unit included in the aerial floating image display device forms the aerial floating image in an opening on the front surface of a housing of the product vending machine or the ticket vending machine. The aerial floating image is formed at a position corresponding to a position of the front surface in the depth direction, or at a position extending forward from the position of the front surface. Aerial floating image display device.

10. In the aerial floating image display device according to claim 1, At a first point in time, the aerial floating image display device displays a first image that accepts an operation in a direction within a plane of the aerial floating image, prompts a user to perform an operation on the first image, and causes the user to make a first selection decision in the predetermined process. After the first selection decision, at a second point in time, the aerial floating image display device displays a second image for selection decision in the aerial floating image, prompts the user to perform an operation on the second image, and causes the user to make a second selection decision. Aerial floating image display device.

11. In the aerial floating image display device according to claim 1, The operation unit includes a partition for physically dividing a plurality of object images displayed in the aerial floating image on the front surface of a housing of the product vending machine or the ticket vending machine. Aerial floating image display device.

12. In the aerial floating image display device according to claim 1, The operation unit includes a plurality of openings for physically dividing a plurality of object images displayed in the airborne floating image on the front surface of the housing of the vending machine or the ticket vending machine. Airborne floating image display device.

13. In the airborne floating image display device according to claim 11, The partition is connected via a housing connection part at a position protruding forward from the front surface of the housing of the vending machine or the ticket vending machine. Airborne floating image display device.

14. In the airborne floating image display device according to claim 11, The partition is arranged at a position in the depth direction corresponding to the formation position of the airborne floating image with respect to the front surface of the housing of the vending machine or the ticket vending machine. Airborne floating image display device.

15. In the airborne floating image display device according to claim 11, The partition is arranged at a position deeper than the formation position of the airborne floating image in the depth direction with respect to the front surface of the housing of the vending machine or the ticket vending machine. Airborne floating image display device.

16. In the airborne floating image display device according to claim 11, The partition is arranged at a position closer to the front than the formation position of the airborne floating image in the depth direction with respect to the front surface of the housing of the vending machine or the ticket vending machine. Airborne floating image display device.

17. In the airborne floating image display device according to claim 1, The predetermined process includes processes related to ticket selection and ticket issuance of the ticket vending machine. The predetermined process includes a process of selecting one item or button from a plurality of items or buttons on the display of the ticket vending machine. Airborne floating image display device.

18. In the airborne floating image display device according to claim 17, The operation unit includes a partition for physically dividing a plurality of object images displayed in the airborne floating image on the front surface of the housing of the ticket vending machine. Airborne floating image display device.

19. In the airborne floating image display device according to claim 1, The predetermined process includes processes related to ticket selection and ticket issuance of the ticket vending machine. The operation unit displays a screen having a plurality of items or buttons for operations related to ticket selection and ticket issuance of the ticket vending machine in the airborne floating image. Based on the detection of an operation on the screen of the floating image, as the predetermined process, execute a process of selecting one item or button from the plurality of items or buttons. Floating image display device.

20. In the floating image display device according to claim 18, The operation unit includes a partition for physically dividing a plurality of object images displayed in the floating image on the front surface of the housing of the vending machine. Floating image display device.

21. A vending machine, Comprising a floating image display device mounted on the housing of the vending machine, The floating image display device includes: An image processing unit that performs image processing; A display unit that displays the image processed by the image processing unit; An optical system that generates a floating image based on the image displayed by the display unit; A user operation detection mechanism that detects an operation by a user on the display range of the floating image; And is provided with, The floating image display device constitutes an operation unit for operating the vending machine, The operation unit displays the floating image for operating the vending machine, Based on the detection of an operation on the floating image, the operation unit causes the vending machine to execute a predetermined process, The predetermined process includes a process of selecting one from a plurality of candidates, The floating image includes an object image that receives an operation for the selection. Vending machine.

22. A display device having an automatic ticket selling function, Comprising a floating image display device mounted on the housing of the display device, The floating image display device includes: An image processing unit that performs image processing; A display unit that displays the image processed by the image processing unit; An optical system that generates a floating image based on the image displayed by the display unit; A user operation detection mechanism that detects an operation by a user on the display range of the floating image; And is provided with, The floating image display device constitutes an operation unit for operating the display device, The operation unit displays the floating image for operating the display device, Based on the detection of an operation on the floating image, the operation unit causes the display device to execute a predetermined process, The predetermined process includes a process of selecting one from a plurality of candidates, The floating image includes an object image that receives an operation for the selection. Display device.

Images