Aerial suspension image display device

Abstract

The invention provides a better aerial suspension image display device. According to the invention, it is possible to contribute to "3 good health and welfare", "9 industry, innovation and infrastructure", "11 sustainable cities and communities" of sustainable development objectives (SDGs). In the aerial floating image display device, image light emitted from a display unit forms a real aerial floating image at a predetermined position in the air through a polarization separation member, a lambda / 4 wave plate, and a retro-reflection plate, the aerial floating image is formed into front and rear two layers of aerial floating images in the depth direction, the front layer is referred to as a first aerial floating image, and the rear layer is referred to as a second aerial floating image. A first object image is displayed in a first aerial suspension image, a rear layer is called a second aerial suspension image, and the first object image is displayed in the first aerial suspension image when an aerial operation performed by a user on the first object image of the first aerial suspension image is detected. Control is performed to change display content of a first object image of the first aerial suspension image and to display a second object image corresponding to the first object image in the second aerial suspension image.

Description

Technical Field

[0001] This invention relates to an aerial levitation image display device. Background Technology

[0002] Regarding the technology of displaying information suspended in the air, for example, it is disclosed in Patent Document 1.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-128722 Summary of the Invention

[0006] The technical problem that the invention aims to solve

[0007] However, the content disclosed in Patent Document 1 does not fully consider how to obtain practical brightness and quality of aerial levitation images, or how to make users enjoy watching aerial levitation images more pleasantly.

[0008] The purpose of this invention is to provide a better aerial levitation image display device.

[0009] Technical means to solve the problem

[0010] To solve the above problems, for example, the structure described in the claimed technical solution is adopted. This application includes various means to solve the above problems, one example of which is as follows. An aerial levitation image display device for displaying aerial levitation images includes: a display unit for displaying images; a polarization separation component; a λ / 4 waveplate; a retroreflector; and a sensor for detecting aerial operations performed by a user on the aerial levitation image, wherein image light emitted from the display unit corresponding to the image displayed on the display unit passes through the polarization separation component, the λ / 4 waveplate, and the retroreflector to form a real image of the aerial levitation image at a predetermined position in the air, the aerial levitation image is formed as two layers of aerial levitation images in the depth direction when viewed from the user's viewpoint, the front layer is referred to as a first aerial levitation image, and the rear layer is referred to as a second aerial levitation image, a first object image is displayed in the first aerial levitation image, an aerial operation performed by the user on the first object image of the first aerial levitation image is detected, and when the aerial operation is detected, control is performed to change the display content of the first object image of the first aerial levitation image and a second object image corresponding to the first object image is displayed in the second aerial levitation image.

[0011] Invention Effects

[0012] The present invention enables the realization of a better aerial levitation image display device. Other technical problems, technical features, and technical effects will become clear in the following description of the embodiments. Attached Figure Description

[0013] Figure 1 This figure shows an example of the usage mode of a spatial levitation image display device according to an embodiment of the present invention.

[0014] Figure 2A This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0015] Figure 2B This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0016] Figure 2C This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0017] Figure 3 This is a diagram illustrating a structural example of a spatial levitation image display device according to an embodiment of the present invention.

[0018] Figure 4A This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0019] Figure 4B This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0020] Figure 4C This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0021] Figure 4D This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0022] Figure 4E This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0023] Figure 4F This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0024] Figure 4G This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0025] Figure 4H This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0026] Figure 4I This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0027] Figure 4J This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0028] Figure 4K This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0029] Figure 4L This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0030] Figure 4M This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0031] Figure 5 This is a cross-sectional view illustrating an example of the specific structure of a light source device according to an embodiment of the present invention.

[0032] Figure 6 This is a cross-sectional view illustrating an example of the specific structure of a light source device according to an embodiment of the present invention.

[0033] Figure 7 This is a cross-sectional view illustrating an example of the specific structure of a light source device according to an embodiment of the present invention.

[0034] Figure 8 This is a configuration diagram showing the main parts of a spatial levitation image display device according to an embodiment of the present invention.

[0035] Figure 9 This is a cross-sectional view showing the structure of a display device according to an embodiment of the present invention.

[0036] Figure 10 This is a cross-sectional view showing the structure of a display device according to an embodiment of the present invention.

[0037] Figure 11 This is an explanatory diagram illustrating the light source diffusion characteristics of an image display device according to an embodiment of the present invention.

[0038] Figure 12 This is an explanatory diagram illustrating the diffusion characteristics of an image display device according to an embodiment of the present invention.

[0039] Figure 13AThis is an illustrative diagram illustrating an example of the technical problem to be solved by image processing according to an embodiment of the present invention.

[0040] Figure 13B This is an illustrative diagram illustrating an example of image processing according to an embodiment of the present invention.

[0041] Figure 13C This is an illustrative diagram illustrating an example of image display processing according to an embodiment of the present invention.

[0042] Figure 13D This is an illustrative diagram illustrating an example of image display processing according to an embodiment of the present invention.

[0043] Figure 14 This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0044] Figure 15A This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0045] Figure 15B This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0046] Figure 16A This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0047] Figure 16B This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0048] Figure 17A This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0049] Figure 17B This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0050] Figure 17C This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0051] Figure 17D This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0052] Figure 18AThis is a diagram illustrating an example of a spatial levitation image display device according to an embodiment of the present invention.

[0053] Figure 18B This is a diagram illustrating an example of a spatial levitation image display device according to an embodiment of the present invention.

[0054] Figure 18C This is a diagram illustrating an example of a spatial levitation image display device according to an embodiment of the present invention.

[0055] Figure 18D This is a diagram illustrating an example of a spatial levitation image display device according to an embodiment of the present invention.

[0056] Figure 18E This is a diagram illustrating an example of a spatial levitation image display device according to an embodiment of the present invention.

[0057] Figure 19A This is a diagram illustrating a display example of a spatial levitation image display device according to an embodiment of the present invention.

[0058] Figure 19B This is a diagram illustrating a display control example of a spatial levitation image display device according to an embodiment of the present invention.

[0059] Figure 19C This is a diagram illustrating a display control example of a spatial levitation image display device according to an embodiment of the present invention.

[0060] Figure 19D This is a diagram illustrating a display control example of a spatial levitation image display device according to an embodiment of the present invention.

[0061] Figure 19E This is a diagram illustrating a display example of a spatial levitation image display device according to an embodiment of the present invention.

[0062] Figure 19F This is a diagram illustrating a processing example of a spatial levitation image display device according to an embodiment of the present invention.

[0063] Figure 19G This is a diagram illustrating an operational example of a spatial levitation image display device according to an embodiment of the present invention.

[0064] Figure 19H This is a diagram illustrating a processing example of a spatial levitation image display device according to an embodiment of the present invention.

[0065] Figure 19I This is a diagram illustrating a display control example of a spatial levitation image display device according to an embodiment of the present invention.

[0066] Figure 19JThis is a diagram illustrating a display control example of a spatial levitation image display device according to an embodiment of the present invention.

[0067] Figure 20A This is a diagram illustrating a structural example of a spatial levitation image display device according to an embodiment of the present invention.

[0068] Figure 20B This is a diagram illustrating an example of the structure of a sensor in a spatial levitation image display device according to an embodiment of the present invention.

[0069] Figure 20C This is a diagram illustrating a display example of a spatial levitation image display device according to an embodiment of the present invention.

[0070] Figure 20D This is a diagram illustrating an example of the arrangement of objects in a spatial levitation image display device according to an embodiment of the present invention.

[0071] Figure 20E This is a diagram illustrating a display example of a spatial levitation image display device according to an embodiment of the present invention.

[0072] Figure 20F This is a diagram illustrating a display example of a spatial levitation image display device according to an embodiment of the present invention.

[0073] Figure 20G This is a diagram illustrating a display example of a spatial levitation image display device according to an embodiment of the present invention.

[0074] Figure 20H This is a diagram illustrating a display example of a spatial levitation image display device according to an embodiment of the present invention.

[0075] Figure 21A This is a diagram illustrating a structural example of a spatial levitation image display device according to an embodiment of the present invention.

[0076] Figure 21B This is a diagram illustrating a display example of a spatial levitation image display device according to an embodiment of the present invention.

[0077] Figure 21C This is a diagram illustrating a display example of a spatial levitation image display device according to an embodiment of the present invention.

[0078] Figure 21D This is a diagram illustrating an example of the arrangement of objects in a spatial levitation image display device according to an embodiment of the present invention. Detailed Implementation

[0079] The embodiments of the present invention are described in detail below based on the accompanying drawings. However, the present invention is not limited to the description of the embodiments, and those skilled in the art can make various changes and modifications within the scope of the technical concept disclosed in this specification. Furthermore, in all the drawings used to illustrate the present invention, parts having the same function are labeled with the same reference numerals, and sometimes repeated descriptions are omitted.

[0080] The following embodiments relate to an image display device that enables an image formed by image light from an image light source to be transmitted through a transparent component such as glass used to separate space, and displayed outside the transparent component as a spatially suspended image. Furthermore, in the following description of the embodiments, the term "spatially suspended image" is used to describe an image suspended in space. Alternatively, it can be expressed as "aerial image," "spatial image," "aerially suspended image," "spatially suspended optical image displaying an image," "aerially suspended optical image displaying an image," etc. The term "spatially suspended image" primarily used in the description of the embodiments is taken as a representative example of these terms.

[0081] According to the following embodiments, a good image display device can be implemented in, for example, bank ATMs, station ticket machines, digital signage, etc. For example, while touch panels are commonly used in bank ATMs and station ticket machines, transparent glass or light-transmitting panels can also be used to display high-resolution image information in a spatially suspended state. In this case, by reducing the divergence angle of the emitted image light to an acute angle, and then unifying it to a specific polarization, only the light normally reflected by the retroreflector can be efficiently reflected. Therefore, light utilization efficiency is high, and ghosting, which is a problem in existing retroreflection methods besides the main spatially suspended image, can be suppressed, resulting in a clear spatially suspended image. Furthermore, by using a device including the light source of this embodiment, a novel and highly usable spatially suspended image display device (spatially suspended image display system) that significantly reduces power consumption can be provided. Additionally, for example, a vehicle-use spatially suspended image display device capable of displaying so-called one-way spatially suspended images can be provided, which can be viewed from inside and / or outside the vehicle.

[0082] <Example 1>

[0083] <An example of the usage of a spatial levitation image display device>

[0084] Figure 1Figure 2 shows an example of the usage mode of the spatial levitation image display device according to an embodiment of the present invention, and also shows the overall structure of the spatial levitation image display device of this embodiment. The specific structure of the spatial levitation image display device will be described in detail using Figure 2 and the like. In this device, light with narrow-angle pointing characteristics and specific polarization emitted from the image display device 1 is used as an image beam. After reflection by the optical system within the spatial levitation image display device, the light first enters the retroreflector plate 2. After retroreflection, the light passes through the transparent component 100 (glass, etc.) and forms a real aerial image (spatial levitation image 3) on the outer side of the glass surface. In addition, in the following embodiments, the retroreflector plate 2 (retroreflective reflector) is used as an example of a retroreflective component. However, the retroreflector plate 2 of the present invention is not limited to a planar plate. It is used as an example, and its concept includes a sheet-like retroreflective component that can be attached to a planar or non-planar component, and an entire assembly obtained by attaching a sheet-like retroreflective component to a planar or non-planar component.

[0085] Furthermore, in shops and similar establishments, display windows (also known as "window glass") 105, constructed from translucent components such as glass, divide the space. The spatial levitation image display device according to this embodiment can display levitation images unidirectionally to the exterior and / or interior of the shop (space) through these transparent components.

[0086] Figure 1 In this context, the inner side of the window glass 105 (inside the shop) is considered as the depth direction, and its outer side (e.g., the sidewalk) as the near side. On the other hand, light can also be reflected by providing a mechanism on the window glass 105 that performs specific polarization reflection, thus forming an aerial image at a desired location inside the shop.

[0087] <Example of the structure of the optical system of a spatial levitation image display device>

[0088] Figure 2A This diagram illustrates an example of the structure of the optical system of a spatial levitation image display device according to an embodiment of the present invention. Figure 2A To explain the structure of the spatial levitation image display device in more detail. For example... Figure 2A As shown in (1), a display device 1 is provided in the oblique direction of a transparent component 100 such as glass, which diffuses image light of a specific polarization in a narrow angle. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates light of a specific polarization with narrow angle diffusion characteristics.

[0089] Image light of a specific polarization from display device 1 is reflected by polarization separation member 101 (in the figure, polarization separation member 101 is formed as a sheet and attached to transparent member 100) which has a film that selectively reflects the image light of the specific polarization and is disposed on transparent member 100, and then incident on retroreflector 2. A λ / 4 waveplate 21 is disposed on the image light incident surface of retroreflector 2. The image light passes through the λ / 4 waveplate 21 twice, once when incident on retroreflector 2 and once when exiting, thereby undergoing polarization transformation from a specific polarization to another polarization. Here, the polarization separation member 101, which selectively reflects the image light of the specific polarization, has the property of transmitting light of the other polarization after polarization transformation, so the image light of the specific polarization after polarization transformation passes through polarization separation member 101. The image light after passing through polarization separation member 101 forms a spatially suspended image 3 of real image on the outside of transparent member 100.

[0090] This section explains Figure 2A This is the first example of polarization design in an optical system. For instance, it can be configured such that S-polarized image light is emitted from a display device 1 to a polarization separation member 101, which has the characteristic of reflecting S-polarized light and transmitting P-polarized light. In this case, the S-polarized image light arriving at the polarization separation member 101 from the display device 1 is reflected by the polarization separation member 101 and goes to the retroreflector 2. When the image light is reflected on the retroreflector 2, it passes twice through the λ / 4 waveplate 21 disposed on the incident surface of the retroreflector 2, so the image light is converted from S-polarized light to P-polarized light. The image light converted to P-polarized light goes to the polarization separation member 101 again. Here, since the polarization separation member 101 has the characteristic of reflecting S-polarized light and transmitting P-polarized light, the P-polarized image light passes through the polarization separation member 101 and through the transparent member 100. The image light passing through the transparent component 100 is generated by the retroreflector 2. Therefore, at the position where the polarization separation component 101 and the displayed image of the display device 1 are in a mirror relationship, an optical image of the displayed image of the display device 1, namely the spatially suspended image 3, is formed. This polarization design enables the spatially suspended image 3 to be formed well.

[0091] Next, the explanation Figure 2AThis is a second example of polarization design in an optical system. For instance, it can be configured such that P-polarized image light is emitted from display device 1 to polarization separation member 101, which has the characteristic of reflecting P-polarized light and transmitting S-polarized light. In this case, the P-polarized image light arriving at polarization separation member 101 from display device 1 is reflected by polarization separation member 101 and goes to retroreflector 2. When the image light is reflected on retroreflector 2, it passes twice through λ / 4 waveplate 21 disposed on the incident surface of retroreflector 2, so the image light is converted from P-polarized light to S-polarized light. The image light converted to S-polarized light goes to polarization separation member 101 again. Here, since polarization separation member 101 has the characteristic of reflecting P-polarized light and transmitting S-polarized light, S-polarized image light passes through polarization separation member 101 and through transparent member 100. The image light passing through the transparent component 100 is generated by the retroreflector 2. Therefore, at the position where the polarization separation component 101 and the displayed image of the display device 1 are in a mirror relationship, an optical image of the displayed image of the display device 1, namely the spatially suspended image 3, is formed. This polarization design enables the spatially suspended image 3 to be formed well.

[0092] Furthermore, the light forming the spatial levitation image 3 is a collection of rays converged from the retroreflector 2 to the optical image of the spatial levitation image 3. These rays continue to travel in a straight line after passing through the optical image of the spatial levitation image 3. Therefore, the spatial levitation image 3 differs from the diffused image light formed on a screen using a conventional projector; it is a highly directional image. Thus, in Figure 2A In this structure, when a user views from the direction of arrow A, the spatially suspended image 3 can be seen as a bright image. However, when other people view from the direction of arrow B, the spatially suspended image 3 cannot be seen as an image at all. This feature is very suitable for systems that display images requiring high security, and systems that display highly confidential images that need to be kept secret from people facing the user.

[0093] Furthermore, depending on the performance of the retroreflector 2, the polarization axis of the reflected image light may not be uniform. Additionally, the reflection angle may also be inconsistent. Such inconsistent light may not maintain the intended polarization state and travel angle. For example, light deviating from the intended polarization state and travel angle may directly re-enter the image display surface of the liquid crystal display panel 11 from the position of the retroreflector 2 without passing through the polarization separation component. After being reflected by components within the spatial levitation image display device, this light deviating from the intended polarization state and travel angle may re-enter the image display surface of the liquid crystal display panel 11. This re-entry of the image display surface of the liquid crystal display panel 11 is reflected again on the image display surface constituting the display device 1, potentially causing ghosting and reducing the image quality of the spatial levitation image. Therefore, in this embodiment, an absorptive polarizer 12 can be provided on the image display surface of the display device 1. By allowing the image light emitted from the display device 1 to pass through the absorptive polarizer 12, and by using the absorptive polarizer 12 to absorb the reflected light returning from the polarization separation member 101, the aforementioned re-reflection can be suppressed. This prevents image quality degradation caused by ghosting of spatially suspended images. Specifically, if a structure is adopted in which S-polarized image light is emitted from the display device 1 to the polarization separation member 101, then the absorptive polarizer 12 can be a polarizer that absorbs P-polarized light. Alternatively, if a structure is adopted in which P-polarized image light is emitted from the display device 1 to the polarization separation member 101, then the absorptive polarizer 12 can be a polarizer that absorbs S-polarized light.

[0094] The polarization separation component 101 described above can be formed, for example, by a reflective polarizer or a multilayer metal film that reflects a specific polarization.

[0095] Next, in Figure 2A In (2), the representative retroreflector 2 represents the surface shape of the retroreflector manufactured by Carbide Industries, Ltd. of Japan used in this study. It is composed of regularly arranged hexagonal prisms. The light incident into the interior is reflected on the walls and bottom of the hexagonal prisms to become retroreflected light, which is emitted in the direction corresponding to the incident light, and displays a spatial levitation image based on the image displayed on the display device 1.

[0096] The resolution of this spatial levitation image depends not only on the resolution of the liquid crystal display panel 11, but also significantly on... Figure 2A(2) shows the shape D and spacing P of the retroreflective portion of the retroreflective plate 2. For example, when using a 7-inch WUXGA (1920×1200 pixels) liquid crystal display panel, even if one pixel (one triplet) is about 80μm, if the diameter D of the retroreflective portion is 240μm and the spacing is 300μm, then one pixel of the spatial levitation image is equivalent to 300μm. Therefore, the effective resolution of the spatial levitation image is reduced to about 1 / 3.

[0097] Therefore, in order to make the resolution of the spatially suspended image the same as that of the display device 1, it is preferable to make the diameter and spacing of the retroreflective portion close to one pixel of the liquid crystal display panel. On the other hand, in order to suppress moiré patterns caused by the pixels of the retroreflective plate and the liquid crystal display panel, the spacing ratio of each can be designed to deviate from one pixel by an integer multiple. In addition, the shape can be configured such that neither side of the retroreflective portion coincides with either side of one pixel of the liquid crystal display panel.

[0098] Furthermore, the surface shape of the retroreflector in this embodiment is not limited to the examples described above. Various surface shapes capable of achieving retroreflective properties can be used. Specifically, a retroreflective element obtained by periodically arranging triangular pyramidal prisms, hexagonal pyramidal prisms, other polygonal prisms, or combinations thereof on the surface of the retroreflector in this embodiment can be provided. Alternatively, a retroreflective element formed by periodically arranging these prisms and creating cubic angles can also be provided on the surface of the retroreflector in this embodiment. Alternatively, a capsule lens-type retroreflective element obtained by periodically arranging glass microspheres on the surface of the retroreflector in this embodiment can also be provided. Detailed structures of these retroreflective elements can be derived using existing technology, so detailed descriptions are omitted. Specifically, technologies disclosed in Japanese Patent Application Publication Nos. 2001-33609, 2001-264525, 2005-181555, 2008-70898, and 2009-229942 can be used.

[0099] <Example 1 of other structural features of the optical system of a spatial levitation image display device>

[0100] For other structural examples of the optical system of a spatial levitation image display device, using Figure 2B Please provide an explanation. Additionally... Figure 2B The text is marked with the same meaning. Figure 2A The same reference numerals have the same structure as Figure 2A Same function and structure. To simplify the explanation, repeated descriptions of such structures are omitted.

[0101] Figure 2B optical system and Figure 2ASimilarly, image light with a specific polarization is output from display device 1. This image light with a specific polarization output from display device 1 is input to polarization separation component 101B. Polarization separation component 101B is a component that selectively transmits image light with a specific polarization. Polarization separation component 101B and... Figure 2A Unlike the transparent component 100, the polarization separating component 101B is not integrally formed with the transparent component 100 but is independently plate-shaped. Therefore, the polarization separating component 101B can also be described as a polarization separating plate. For example, the polarization separating component 101B can be configured as a reflective polarizer formed by attaching a polarization separating sheet to the transparent component. Alternatively, it can be formed on the transparent component using a multilayer metal film or the like that selectively transmits specific polarizations and reflects other specific polarizations. Figure 2B In this configuration, the polarization separation component 101B is configured to transmit image light of a specific polarization output from the display device 1.

[0102] Image light passing through polarization separation component 101B is incident on retroreflector 2. A λ / 4 waveplate 21 is provided on the image light incident surface of retroreflector. The image light passes through the λ / 4 waveplate 21 twice, once when it is incident on the retroreflector and once when it is emitted, thereby undergoing polarization transformation from a specific polarization to another polarization. Here, polarization separation component 101B has the property of reflecting the polarized light of the other polarization after polarization transformation by λ / 4 waveplate 21, so the polarization-transformed image light is reflected on polarization separation component 101B. The image light reflected on polarization separation component 101B passes through transparent component 100, forming a spatially suspended real image 3 on the outside of transparent component 100.

[0103] This section explains Figure 2BThis is the first example of polarization design in an optical system. For instance, it can be configured such that P-polarized image light is emitted from display device 1 to polarization separation member 101B, which has the characteristic of reflecting S-polarized light and transmitting P-polarized light. In this case, the P-polarized image light arriving at polarization separation member 101B from display device 1 passes through polarization separation member 101B and goes to retroreflector 2. When the image light is reflected on retroreflector 2, it passes twice through λ / 4 waveplate 21 disposed on the incident surface of retroreflector 2, so the image light is converted from P-polarized light to S-polarized light. The image light converted to S-polarized light goes to polarization separation member 101B again. Here, since polarization separation member 101B has the characteristic of reflecting S-polarized light and transmitting P-polarized light, the S-polarized image light is reflected on polarization separation member 101B and passes through transparent member 100. The image light passing through the transparent component 100 is generated by the retroreflector 2. Therefore, at the position where the polarization separation component 101B and the displayed image of the display device 1 are in a mirror relationship, an optical image of the displayed image of the display device 1, namely the spatially suspended image 3, is formed. This polarization design enables the spatially suspended image 3 to be formed well.

[0104] Next, the explanation Figure 2B This is a second example of polarization design in an optical system. For instance, it can be configured such that S-polarized image light is emitted from display device 1 to polarization separation member 101B, which has the characteristic of reflecting P-polarized light and transmitting S-polarized light. In this case, the S-polarized image light arriving at polarization separation member 101B from display device 1 passes through polarization separation member 101B and goes to retroreflector 2. When the image light is reflected on retroreflector 2, it passes twice through λ / 4 waveplate 21 disposed on the incident surface of retroreflector 2, so the image light is converted from S-polarized light to P-polarized light. The image light converted to P-polarized light goes to polarization separation member 101B again. Here, since polarization separation member 101B has the characteristic of reflecting P-polarized light and transmitting S-polarized light, the P-polarized image light is reflected on polarization separation member 101B and passes through transparent member 100. The image light passing through the transparent component 100 is generated by the retroreflector 2. Therefore, at the position where the polarization separation component 101B and the displayed image of the display device 1 are in a mirror relationship, an optical image of the displayed image of the display device 1, namely the spatially suspended image 3, is formed. This polarization design enables the spatially suspended image 3 to be formed well.

[0105] in addition, Figure 2BIn this configuration, the image display surface of the display device 1 and the surface of the retroreflector 2 are arranged parallel to each other. The polarization separation member 101B is configured to be tilted at an angle α (e.g., 30°) relative to the image display surface of the display device 1 and the surface of the retroreflector 2. Thus, when reflected at the polarization separation member 101B, the direction of travel of the image light reflected at the polarization separation member 101B (the direction of the principal ray of the image light) differs from the direction of travel of the image light incident from the retroreflector 2 by an angle β (e.g., 60°). By employing this structure, in... Figure 2B In the optical system, image light is output to the outside of the transparent component 100 at a specified angle as shown in the figure, forming a spatially suspended image 3 of real image. Figure 2B In this structure, when a user views from the direction of arrow A, the spatially suspended image 3 can be seen as a bright image. However, when other people view from the direction of arrow B, the spatially suspended image 3 cannot be seen as an image at all. This feature is very suitable for systems that display images requiring high security, and systems that display highly confidential images that need to be kept secret from people facing the user.

[0106] As explained above, Figure 2B The optical system is with Figure 2A Optical systems with different structures, but capable of operating with... Figure 2A The optical system also forms a good spatial levitation image.

[0107] Additionally, an absorptive polarizer can be provided on the surface of the transparent component 100 on the side of the polarization separation member 101B. This absorptive polarizer can be one that allows the polarized transmission of image light from the polarization separation member 101B while absorbing polarizations that are 90° out of phase with the image light from the polarization separation member 101B. This allows for sufficient transmission of image light used to form the spatial levitation image 3, while reducing external light incident from the spatial levitation image 3 side of the transparent component 100 by approximately 50%. Consequently, the amount of light generated by external light incident from the spatial levitation image 3 side of the transparent component 100 can be reduced. Figure 2B Stray light within the optical system.

[0108] <Example 2 of other structural features of the optical system of a spatial levitation image display device>

[0109] For other structural examples of the optical system of a spatial levitation image display device, using Figure 2C Please provide an explanation. Additionally... Figure 2C The text is marked with the same meaning. Figure 2B The same reference numerals have the same structure as Figure 2B Same function and structure. To simplify the explanation, repeated descriptions of such structures are omitted.

[0110] Figure 2C Optical system relative to Figure 2B The only difference in the optical system is the arrangement angle of the polarization separation component 101B relative to the image display surface of the display device 1 and the surface of the retroreflector 2. All other structures are the same. Figure 2B Since the optical systems are the same, repeated descriptions are omitted. Figure 2C The polarization design of the optical system is also related to Figure 2B Since the polarization design of the optical system is the same, repeated explanations are omitted.

[0111] exist Figure 2C In the optical system, the polarization separation component 101B is configured to be tilted at an angle α relative to the image display surface of the display device 1 and the surface of the retroreflector 2. Figure 2C In this configuration, the angle α is 45°. With this structure, when reflected from the polarization separating member 101B, the angle β between the direction of travel of the image light incident from the retroreflector 2 (the direction of the principal ray of the image light) and the direction of travel of the image light reflected from the polarization separating member 101B (the direction of the principal ray of the image light) is 90°. With this structure, the image display surface of the display device 1 and the surface of the retroreflector 2 are perpendicular to the direction of travel of the image light reflected from the polarization separating member 101B, simplifying the angular relationships of the surfaces constituting the optical system. If the surface of the transparent member 100 is configured to be orthogonal to the direction of travel of the image light reflected from the polarization separating member 101B, the angular relationships of the surfaces constituting the optical system can be further simplified. Figure 2C In this structure, when a user views from the direction of arrow A, the spatially suspended image 3 can be seen as a bright image. However, when other people view from the direction of arrow B, the spatially suspended image 3 cannot be seen as an image at all. This feature is very suitable for systems that display images requiring high security, and systems that display highly confidential images that need to be kept secret from people facing the user.

[0112] As explained above, Figure 2C The optical system is with Figure 2A and Figure 2B Optical systems with different structures, but capable of operating with... Figure 2A and Figure 2B The optical system also produces excellent spatial levitation images. Furthermore, it simplifies the angles of the surfaces that constitute the optical system.

[0113] Additionally, an absorptive polarizer can be provided on the surface of the transparent component 100 on the side of the polarization separation member 101B. This absorptive polarizer can be one that allows the polarized transmission of image light from the polarization separation member 101B while absorbing polarizations that are 90° out of phase with the image light from the polarization separation member 101B. This allows for sufficient transmission of image light used to form the spatial levitation image 3, while reducing external light incident from the spatial levitation image 3 side of the transparent component 100 by approximately 50%. Consequently, the amount of light generated by external light incident from the spatial levitation image 3 side of the transparent component 100 can be reduced. Figure 2C Stray light within the optical system.

[0114] Based on the above explanation Figure 2A , Figure 2B , Figure 2C Its optical system can provide brighter, higher-quality images of spatial levitation.

[0115] <<Block diagram of the internal structure of the spatial levitation image display device>>

[0116] Next, a block diagram of the internal structure of the spatial levitation image display device 1000 will be described. Figure 3 This is a block diagram illustrating an example of the internal structure of a spatial levitation image display device 1000.

[0117] The spatial levitation image display device 1000 includes a retroreflective unit 1101, an image display unit 1102, a light guide 1104, a light source 1105, a power supply 1106, an external power input interface 1111, an operation input unit 1107, a non-volatile memory 1108, a memory 1109, a control unit 1110, an image signal input unit 1131, an audio signal input unit 1133, a communication unit 1132, an air operation detection sensor 1351, an air operation detection unit 1350, an audio output unit 1140, an image control unit 1160, a storage unit 1170, and a camera unit 1180. Additionally, it may also include a movable media interface 1134, an attitude sensor 1113, a transmissive self-emissive image display device 1650, a second display device 1680, or a secondary battery 1112.

[0118] The various components of the spatial levitation image display device 1000 are arranged within the housing 1190. Furthermore, Figure 3 The camera unit 1180 and the air operation detection sensor 1351 shown can also be located on the outside of the housing 1190.

[0119] Figure 3 The return reflection section 1101 corresponds to Figure 2A , Figure 2B , Figure 2CThe retroreflector 2. The retroreflector 1101 causes the light modulated by the image display unit 1102 to undergo retroreflection. The light reflected from the retroreflector 1101 and output to the outside of the spatial levitation image display device 1000 forms a spatial levitation image 3.

[0120] Figure 3 The image display unit 1102 corresponds to Figure 2A , Figure 2B , Figure 2C LCD display panel 11. Figure 3 The light source 1105 corresponds to Figure 2A , Figure 2B , Figure 2C The light source device 13. Figure 3 The image display unit 1102, light guide 1104, and light source 1105 correspond to Figure 2A , Figure 2B , Figure 2C Display device 1.

[0121] The image display unit 1102 is a display unit that generates an image by modulating transmitted light based on an image signal input under the control of the image control unit 1160 (described later). The image display unit 1102 corresponds to... Figure 2A , Figure 2B , Figure 2C The liquid crystal display panel 11. For example, a transmissive liquid crystal panel is used as the image display unit 1102. Alternatively, a reflective liquid crystal panel that modulates reflected light or a DMD (Digital Micromirror Device) panel can be used as the image display unit 1102.

[0122] The light source 1105 generates light for the image display unit 1102 and is a solid-state light source such as an LED light source or a laser light source. The power supply 1106 converts AC current input from the outside via the external power input interface 1111 into DC current to power the light source 1105. Additionally, the power supply 1106 supplies necessary DC current to various parts within the spatial levitation image display device 1000. The secondary battery 1112 stores the power supplied by the power supply 1106. Furthermore, when not powered externally via the external power input interface 1111, the secondary battery 1112 powers the light source 1105 and other structures requiring power. In other words, with the secondary battery 1112 in place, the spatial levitation image display device 1000 can be used by the user even without external power.

[0123] The light guide 1104 guides the light generated by the light source 1105, directing it towards the image display unit 1102. The combination of the light guide 1104 and the light source 1105 can also be referred to as the backlight of the image display unit 1102. The light guide 1104 can be constructed primarily of glass. Alternatively, it can be constructed primarily of plastic. It can also be constructed using a reflector. Various combinations of the light guide 1104 and the light source 1105 are possible. Specific structural examples of the combination of the light guide 1104 and the light source 1105 will be described in detail later.

[0124] The air-to-air operation detection sensor 1351 is a sensor that detects the user 230's finger operation on the spatial levitation image 3. The air-to-air operation detection sensor 1351 can, for example, sense the area overlapping with the entire display area of ​​the spatial levitation image 3. Alternatively, the air-to-air operation detection sensor 1351 can also sense only the area overlapping with at least a portion of the display area of ​​the spatial levitation image 3.

[0125] As a specific example of the airborne operation detection sensor 1351, a distance sensor using non-visible light such as infrared, non-visible light laser, or ultrasound can be cited. Alternatively, the airborne operation detection sensor 1351 can also combine multiple sensors to form a system capable of detecting coordinates in a two-dimensional plane. Furthermore, the airborne operation detection sensor 1351 can be constructed from a ToF (Time of Flight) type LiDAR (Light Detection and Ranging) sensor or an image sensor.

[0126] The airborne operation detection sensor 1351 only needs to be able to sense and detect touch operations performed by the user's finger on an object displayed as a spatial levitation image 3. Such sensing can also be performed using existing technologies.

[0127] The airborne operation detection unit 1350 acquires sensing signals from the airborne operation detection sensor 1351, determines whether the user 230's finger has made contact with an object in the space-suspended image 3 based on the sensing signals, and calculates the position of contact between the user 230's finger and the object (contact position), etc. The airborne operation detection unit 1350 is constructed, for example, by a circuit such as an FPGA (Field Programmable Gate Array). In addition, some functions of the airborne operation detection unit 1350 can also be implemented in software, for example, by a spaceborne operation detection program executed by the control unit 1110.

[0128] The airborne operation detection sensor 1351 and the airborne operation detection unit 1350 can be built into the space-based levitation image display device 1000, or they can be installed separately from the space-based levitation image display device 1000. When installed separately from the space-based levitation image display device 1000, the airborne operation detection sensor 1351 and the airborne operation detection unit 1350 are configured to transmit information and signals to the space-based levitation image display device 1000 via wired or wireless communication connection paths or image signal transmission paths.

[0129] Furthermore, the airborne operation detection sensor 1351 and the airborne operation detection unit 1350 can also be installed separately. This allows for the construction of a system that can optionally add only the airborne operation detection function, using the space-based levitation image display device 1000, which does not have airborne operation detection functionality, as the main body. Alternatively, only the airborne operation detection sensor 1351 can be separated, and the airborne operation detection unit 1350 can be integrated into the space-based levitation image display device 1000. In cases where more flexibility in the placement of the airborne operation detection sensor 1351 relative to the installation location of the space-based levitation image display device 1000 is desired, the structure that separates only the airborne operation detection sensor 1351 has advantages.

[0130] The camera unit 1180 is, for example, a camera with an image sensor, that captures images of the space near the levitation image 3 and / or the user 230's face, arms, fingers, etc. Multiple camera units 1180 can be provided. By using multiple camera units 1180, or by using a camera unit with a depth sensor, the air operation detection unit 1350 can be assisted when detecting touch operations by the user 230 on the levitation image 3. The camera unit 1180 can also be provided separately from the levitation image display device 1000. When the camera unit 1180 is provided separately from the levitation image display device 1000, it can be configured to transmit camera signals to the levitation image display device 1000 via a wired or wireless communication connection path, etc.

[0131] For example, if the airborne operation detection sensor 1351 is configured to detect whether an object has invaded the intrusion detection plane by targeting the plane (intrusion detection plane) that includes the display surface of the spatial levitation image 3, there may be situations where the airborne operation detection sensor 1351 cannot detect how far away an object (such as a user's finger) that has not invaded the intrusion detection plane is from the intrusion detection plane, or how close the object is to the intrusion detection plane.

[0132] In this case, by using depth calculation information of the object obtained from images captured by multiple camera units 1180 and depth information of the object obtained by a depth sensor, the distance between the object and the intrusion detection plane can be calculated. Then, this information and various other information, such as the distance between the object and the intrusion detection plane, are used for various display controls of the spatial levitation image 3.

[0133] Alternatively, instead of using the air operation detection sensor 1351, the air operation detection unit 1350 can detect the user 230's touch operation on the spatial levitation image 3 based on the image captured by the camera unit 1180.

[0134] Alternatively, the camera unit 1180 can capture the face of the user 230 operating the floating image 3, and the control unit 1110 can perform user 230 recognition processing. Furthermore, to determine whether someone is standing around or behind the user 230 operating the floating image 3, or whether someone is spying on the user 230's operation of the floating image 3, the camera unit 1180 can also capture images of the area including the user 230 operating the floating image 3 and the area surrounding the user 230.

[0135] The operation input unit 1107 is, for example, a signal receiving unit or an infrared receiving unit such as an operation button or a remote control, which inputs signals for operations different from the air operation (touch operation) performed by the user 230. In addition to the user 230 who performs touch operation on the spatial levitation image 3, the operation input unit 1107 can also be used for an administrator to operate the spatial levitation image display device 1000.

[0136] The video signal input unit 1131 connects to an external video output device to input video data. Various digital video input interfaces can be considered for the video signal input unit 1131. For example, it can be configured as an HDMI (High-Definition Multimedia Interface) standard video input interface, a DVI (Digital Visual Interface) standard video input interface, or a DisplayPort standard video input interface. Alternatively, analog video input interfaces such as analog RGB and component video can also be provided. The audio signal input unit 1133 connects to an external audio output device to input audio data. The audio signal input unit 1133 can be configured as an HDMI standard audio input interface, an optical digital terminal interface, or a coaxial digital terminal interface. When using an HDMI standard interface, the video signal input unit 1131 and the audio signal input unit 1133 can be configured as an interface integrating terminals and cables. The audio output unit 1140 can output audio based on the audio data input to the audio signal input unit 1133. The audio output unit 1140 can be configured as a speaker. Additionally, the audio output unit 1140 can also output built-in operation tones and error warning tones. Alternatively, the structure that outputs digital signals to external devices, as specified in the HDMI standard's Audio Return Channel function, can be used as the audio output unit 1140.

[0137] The non-volatile memory 1108 stores various data used in the spatial levitation image display device 1000. The data stored in the non-volatile memory 1108 includes, for example, various operational data for displaying the spatial levitation image 3, display icons, data on objects for user operation, and layout information. The main memory 1109 stores image data displayed as the spatial levitation image 3 and control data for the device.

[0138] The control unit 1110 controls the operation of each connected component. In addition, the control unit 1110 can cooperate with the program stored in the memory 1109 to perform calculations based on information obtained from each component in the spatial levitation image display device 1000.

[0139] The communication unit 1132 communicates with external devices, external servers, etc., via a wired or wireless communication interface. When the communication unit 1132 has a wired communication interface, this interface can be configured as, for example, a LAN interface using an Ethernet standard. When the communication unit 1132 has a wireless communication interface, it can be configured as, for example, a Wi-Fi communication interface, a Bluetooth communication interface, or a 4G or 5G mobile communication interface. Through communication via the communication unit 1132, various data such as video data, image data, and audio data are sent and received.

[0140] Additionally, the removable media interface 1134 is an interface for connecting a removable recording medium (removable medium). The removable recording medium (removable medium) can be composed of semiconductor memory such as a solid-state drive (SSD), magnetic recording media such as a hard disk drive (HDD), or optical recording media such as an optical disc. The removable media interface 1134 can read various data and information, such as image data, audio data, etc., recorded on the removable recording medium. The image data and audio data recorded on the removable recording medium are output as a spatial levitation image 3 via the image display unit 1102 and the retroreflective unit 1101.

[0141] Storage unit 1170 is a storage device that records various types of data and information, such as image data, audio data, etc. Storage unit 1170 can be composed of a magnetic recording medium recording device such as a hard disk drive (HDD) or a semiconductor device memory such as a solid-state drive (SSD). In storage unit 1170, for example, various types of data and information, such as image data, audio data, etc., can be pre-recorded at the time of product shipment. Furthermore, storage unit 1170 can also record various types of data and information, such as image data, audio data, etc., obtained from external devices and external servers via communication unit 1132.

[0142] The image data and other data recorded in the storage unit 1170 are output as a spatial levitation image 3 via the image display unit 1102 and the retroreflection unit 1101. The image data and other data of the display icons and objects for user operation displayed as the spatial levitation image 3 are also recorded in the storage unit 1170.

[0143] The layout information of the display icons, objects, etc., displayed as the spatial levitation image 3, as well as various metadata information about the objects, are also recorded in the storage unit 1170. The sound data recorded in the storage unit 1170 is output as sound, for example, from the sound output unit 1140.

[0144] The image control unit 1160 performs various controls on the image signals input to the image display unit 1102. The image control unit 1160 can also be called an image processing circuit, and may be constructed from hardware such as an ASIC, FPGA, or video processor. Alternatively, the image control unit 1160 can also be called an image processing unit or an image processing unit. For example, the image control unit 1160 controls image switching, switching which image signal from the image signal stored in the memory 1109 or the image signal (image data) input to the image signal input unit 1131 is input to the image display unit 1102.

[0145] In addition, the image control unit 1160 can also be controlled to generate a superimposed image signal obtained by superimposing the image signal stored in the memory 1109 and the image signal input from the image signal input unit 1131, and input the superimposed image signal to the image display unit 1102, thereby forming a composite image as a spatial levitation image 3.

[0146] Additionally, the image control unit 1160 can also perform control, and perform image processing on the image signals input from the image signal input unit 1131 and the image signals stored in the memory 1109. Examples of image processing include scaling (enlarging, reducing, and distorting the image), brightness adjustment (changing brightness), contrast adjustment (changing the image's contrast curve), and Retinex processing (decomposing the image into light components and changing the weights of each component).

[0147] In addition, the image control unit 1160 can also perform special effects image processing on the image signal input to the image display unit 1102 to assist the user 230 in-flight operation (touch operation). The special effects image processing is performed, for example, based on the detection results of the touch operation of the user 230 by the in-flight operation detection unit 1350 and the image captured by the camera unit 1180 of the user 230.

[0148] The attitude sensor 1113 is a sensor composed of a gravity sensor, an acceleration sensor, or a combination thereof, capable of detecting the setting attitude of the spatial levitation image display device 1000. The control unit 1110 can control the operation of each connected component based on the attitude detection results of the attitude sensor 1113. For example, if the attitude is detected to be unsuitable for the user, control can be implemented to stop the display of the image on the image display unit 1102 and display an error message to the user. Alternatively, if the attitude sensor 1113 detects a change in the setting attitude of the spatial levitation image display device 1000, control can be implemented to rotate the display direction of the image displayed on the image display unit 1102.

[0149] As explained above, the spatial levitation image display device 1000 is equipped with various functions. However, the spatial levitation image display device 1000 does not need to have all of these functions; it can have any structure as long as it has the function of forming the spatial levitation image 3.

[0150] <Structural Example of a Spatial Suspended Image Display Device>

[0151] Next, a structural example of the spatial levitation image display device will be described. Regarding the layout of the constituent elements of the spatial levitation image display device in this embodiment, various layouts may exist depending on the usage configuration. Hereinafter, [details to be added] Figures 4A to 4M The layout of each item will be explained. Additionally, in Figures 4A to 4M In any of the examples, the thick lines surrounding the space levitation image display device 1000 represent an example of the housing structure of the space levitation image display device 1000.

[0152] Figure 4A This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4A The space-floating image display device 1000 shown is equipped with... Figure 2A The optical system corresponding to the optical system. Figure 4A The spatial levitation image display device 1000 shown is horizontally positioned with the side forming the spatial levitation image 3 facing upwards. That is, Figure 4A In this device, the transparent component 100 of the spatial levitation image display device 1000 is provided on the top surface of the device. The spatial levitation image 3 is formed above the surface of the transparent component 100. The light from the spatial levitation image 3 travels obliquely upwards. With the air operation detection sensor 1351 provided as shown, it is possible to detect the user 230's finger operation on the spatial levitation image 3. Furthermore, the x-direction is the left-right direction as seen by the user, the y-direction is the front-back direction (depth direction) as seen by the user, and the z-direction is the up-down direction (vertical direction). Hereinafter, Figures 4A to 4M The definitions of the x, y, and z directions are the same in all the diagrams, so repeated explanations are omitted.

[0153] Figure 4B This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4B The space-floating image display device 1000 shown is equipped with... Figure 2A The optical system corresponding to the optical system. Figure 4B The spatial levitation image display device 1000 shown is vertically arranged such that the side forming the spatial levitation image 3 faces the front of the spatial levitation image display device 1000 (the direction of the user 230). That is, Figure 4B In this configuration, the transparent component 100 of the spatial levitation image display device is located on the front of the device (to the user 230's direction). The spatial levitation image 3 is formed on the user 230's side, compared to the surface of the transparent component 100 of the spatial levitation image display device 1000. The light from the spatial levitation image 3 travels obliquely upwards. With the air operation detection sensor 1351 arranged as shown, it is possible to detect the user 230's finger operation on the spatial levitation image 3. Here, as... Figure 4B As shown, the air-to-air operation detection sensor 1351 senses the user 230's finger from above and uses the reflection of sensing light from the user's fingernail for touch detection. Generally, fingernails have a higher reflectivity than fingertips, so this structure improves the accuracy of touch detection.

[0154] Figure 4C This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4C The space-floating image display device 1000 shown is equipped with... Figure 2B The optical system corresponding to the optical system. Figure 4C The spatial levitation image display device 1000 shown is horizontally positioned with the side forming the spatial levitation image 3 facing upwards. That is, Figure 4C In this device, the transparent component 100 of the spatial levitation image display device 1000 is disposed on the top surface of the device. The spatial levitation image 3 is formed above the surface of the transparent component 100 of the spatial levitation image display device 1000. The light from the spatial levitation image 3 travels obliquely upward. When the air operation detection sensor 1351 is disposed as shown in the figure, it is possible to detect the operation of the user 230's finger on the spatial levitation image 3.

[0155] Figure 4D This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4D The space-floating image display device 1000 shown is equipped with... Figure 2B The optical system corresponding to the optical system. Figure 4D The spatial levitation image display device 1000 shown is vertically arranged such that the side forming the spatial levitation image 3 faces the front of the spatial levitation image display device 1000 (the direction of the user 230). That is, Figure 4D In this configuration, the transparent component 100 of the spatial levitation image display device 1000 is located on the front of the device (to the user 230's direction). The spatial levitation image 3 is formed on the user 230's side, compared to the surface of the transparent component 100. The light from the spatial levitation image 3 travels obliquely upwards. With the air-operation detection sensor 1351 arranged as shown, it is possible to detect the user 230's finger operation on the spatial levitation image 3. Here, as... Figure 4D As shown, the air-to-air operation detection sensor 1351 senses the user 230's finger from above and uses the reflection of sensing light from the user's fingernail for touch detection. Generally, fingernails have a higher reflectivity than fingertips, so this structure improves the accuracy of touch detection.

[0156] Figure 4E This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4E The space-floating image display device 1000 shown is equipped with... Figure 2C The optical system corresponding to the optical system. Figure 4E The spatial levitation image display device 1000 shown is horizontally positioned with the side forming the spatial levitation image 3 facing upwards. That is, Figure 4EIn this device, the transparent component 100 of the spatial levitation image display device 1000 is disposed on the top surface of the device. The spatial levitation image 3 is formed above the surface of the transparent component 100 of the spatial levitation image display device 1000. The light of the spatial levitation image 3 travels in the upward direction. When the air operation detection sensor 1351 is disposed as shown, it is possible to detect the operation of the user 230's finger on the spatial levitation image 3.

[0157] Figure 4F This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4F The space-floating image display device 1000 shown is equipped with... Figure 2C The optical system corresponding to the optical system. Figure 4F The spatial levitation image display device 1000 shown is vertically arranged such that the side forming the spatial levitation image 3 faces the front of the spatial levitation image display device 1000 (the direction of the user 230). That is, Figure 4F In the spatial levitation image display device 1000, the transparent component 100 is disposed on the front of the device (to the user 230). The spatial levitation image 3 is formed on the side of the user 230, compared to the surface of the transparent component 100. The light from the spatial levitation image 3 travels in the direction in front of the user. With the air operation detection sensor 1351 disposed as shown, it is possible to detect the user 230's finger operation on the spatial levitation image 3.

[0158] Figure 4G This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4G The space-floating image display device 1000 shown is equipped with... Figure 2C The optical system corresponding to the optical system. Figures 4A to 4F In the optical system of the spatial levitation image display device, the optical path of the center of the image light emitted from display device 1 lies on the yz plane. That is, in Figures 4A to 4F Within the optical system of the spatial levitation image display device, the image light travels in the front-back and up-down directions as seen from the user's perspective. In contrast, in... Figure 4G In the optical system of the spatial levitation image display device shown, the optical path of the center of the image light emitted from display device 1 lies on the xy plane. That is, in Figure 4G Within the optical system of the spatial levitation image display device shown, the image light travels in the left-right and front-back directions from the user's perspective. Figure 4G The spatial levitation image display device 1000 shown is positioned such that the side forming the spatial levitation image 3 faces the front of the device (the direction of the user 230). That is, Figure 4GIn the spatial levitation image display device 1000, the transparent component 100 is located on the front of the device (to the user 230's direction). The spatial levitation image 3 is formed on the user's side, compared to the surface of the transparent component 100. The light from the spatial levitation image 3 travels in the direction in front of the user. With the air-operation detection sensor 1351 arranged as shown, it is possible to detect the user 230's finger operation on the spatial levitation image 3.

[0159] Figure 4H This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4H Spatial levitation image display device 1000 and Figure 4G The difference in the spatial levitation image display device is that a window is provided on the back of the device (on the opposite side of the position where the user 230 views the spatial levitation image 3, i.e., the opposite side of the direction of travel of the image light of the spatial levitation image 3 towards the user 230), and this window has a transparent plate 100B made of glass or plastic. Other structures are the same as... Figure 4G The spatial levitation image display device has the same structure, so repeated descriptions are omitted. Figure 4H In the spatial levitation image display device 1000, for the spatial levitation image 3, a window with a transparent plate 100B is provided on the opposite side of the direction of travel of the image light of the spatial levitation image 3. Therefore, when the user 230 views the spatial levitation image 3, the scenery behind the spatial levitation image display device 1000 can be identified as the background of the spatial levitation image 3. Thus, the user 230 can perceive that the spatial levitation image 3 is suspended in the air in front of the scenery behind the spatial levitation image display device 1000. This further emphasizes the sense of suspension of the spatial levitation image 3.

[0160] Furthermore, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, there is a possibility that a portion of the image light output from the display device 1 may be reflected by the polarization separation member 101B and go to the transparent plate 100B. And depending on the coating performance of the surface of the transparent plate 100B, there is a possibility that this light may be reflected again on the surface of the transparent plate 100B as stray light and be seen by the user. Therefore, to prevent this stray light, the transparent plate 100B may not be provided on the aforementioned window on the back of the spatially suspended image display device 1000.

[0161] Figure 4I This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4I Spatial levitation image display device 1000 and Figure 4HThe difference in the spatial levitation image display device is that a light-blocking door 1410 is provided at the window of the transparent plate 100B located on the back of the device (on the opposite side of the position where user 230 views the spatial levitation image 3). Other structures are the same as... Figure 4H The spatial levitation image display device has the same structure, so repeated descriptions are omitted. Figure 4I The opening / closing door 1410 of the spatial levitation image display device 1000 has, for example, a light-shielding plate, including a mechanism for moving (sliding) the light-shielding plate, a rotation mechanism, or a mechanism for making the light-shielding plate detachable. This allows the window (rear side window) of the transparent plate 100B located at the depth side of the spatial levitation image display device 1000 to switch between an open state and a light-shielding state. The opening / closing door 1410 can be driven by a motor (not shown) to electrically move (slide) and rotate the light-shielding plate. This motor can be... Figure 3 The control unit 1110 controls this. Additionally... Figure 4I The example disclosed shows that the number of light-shielding panels of the opening and closing door 1410 is 2. However, the number of light-shielding panels of the opening and closing door 1410 can also be 1.

[0162] For example, when the view visible from the depths of the window of the transparent panel 100B of the spatial levitation image display device 1000 is outdoors, the brightness of sunlight varies with the weather. When the outdoor sunlight is strong, the background of the spatial levitation image 3 becomes too bright, reducing the user 230's ability to distinguish the spatial levitation image 3. In such cases, if the back window is shaded by moving (sliding), rotating, or installing the light-shielding plate of the opening / closing door 1410, the background of the spatial levitation image 3 will darken, thus relatively improving the distinguishability of the spatial levitation image 3. This light-shielding action of the light-shielding plate of the opening / closing door 1410 can also be performed manually by the user 230. Alternatively, it can be performed via... Figure 3 The operation input is made by the operation input unit 1107, and the control unit 1110 controls the motor (not shown) to perform the light-blocking action of the light-blocking plate of the door 1410 to open and close accordingly.

[0163] Alternatively, an illuminance sensor can be installed on the rear side of the spatially suspended image display device 1000 (the opposite side of the user 230) – for example, near the rear-side window – to measure the brightness of the space outside the rear-side window. In this case, the brightness can be determined based on the detection results of the illuminance sensor. Figure 3 The control unit 1110 controls a motor (not shown) to open and close the light shield of the door 1410. By controlling the opening and closing of the light shield of the door 1410 in this way, the recognizability of the spatial floating image 3 can be better maintained even if the user 230 does not manually open and close the light shield of the door 1410.

[0164] Furthermore, the light shield of the opening / closing door 1410 can also be manually detachable. Depending on the intended use and environment of the spatial floating image display device 1000, the user can choose whether the rear window is open or blocked. If the rear window is intended to be kept blocked for an extended period, the detachable light shield can be fixed in the blocked state. Alternatively, if the rear window is intended to be kept open for an extended period, the detachable light shield can be removed. The light shield can be attached or removed using screws, hooks, or an embedded structure.

[0165] in addition, Figure 4I The example of the spatial levitation image display device 1000 is similar. Depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, there is a possibility that a portion of the image light output from the display device 1 may be reflected on the polarization separation member 101B and go to the transparent plate 100B. Furthermore, depending on the coating properties of the surface of the transparent plate 100B, there is a possibility that the light may be reflected again on the surface of the transparent plate 100B as stray light and be seen by the user. Therefore, to prevent this stray light, the transparent plate 100B may not be provided on the aforementioned window on the back of the spatial levitation image display device 1000. The aforementioned opening / closing door 1410 may be provided on the window without the transparent plate 100B. To prevent this stray light, the inner surface of the housing of the light-shielding plate of the opening / closing door 1410 preferably has a coating or material with low light reflectivity.

[0166] Figure 4J This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4J Spatial levitation image display device 1000 and Figure 4H The difference in this spatial levitation image display device is that, instead of a transparent glass or plastic panel 100B, it uses an electrically controlled variable transmittance device 1620 for its rear side window. Other structural features are similar to... Figure 4H The structure is the same as that of the spatial levitation image display device, so repeated descriptions are omitted. An example of the electrically controlled variable transmittance device 1620 is a liquid crystal shutter, etc. That is, the liquid crystal shutter can control the transmission of light by controlling the voltage of the liquid crystal element sandwiched between two polarizers. Therefore, if the transmittance of the liquid crystal shutter is increased, the background of the spatial levitation image 3 becomes transparent, allowing the view through the back side window to be seen. Conversely, if the transmittance of the liquid crystal shutter is decreased, the background of the spatial levitation image 3 becomes so transparent that the view through the back side window cannot be seen. Furthermore, the liquid crystal shutter can control intermediate color levels, so a state with a transmittance of 50% or similar can also be used. For example, it can be adjusted according to... Figure 3The operation input unit 1107 performs the operation input, and the control unit 1110 controls the transmittance of the electronically controlled transmittance variable device 1620. With this structure, when viewing the scenery through the back side window as the background of the spatial levitation image 3, but the background, i.e., the scenery through the back side window, is too bright, reducing the recognizability of the spatial levitation image 3, the recognizability of the spatial levitation image 3 can be adjusted by adjusting the transmittance of the electronically controlled transmittance variable device 1620.

[0167] Alternatively, an illuminance sensor can be installed on the rear side of the spatially suspended image display device 1000 (the opposite side of the user 230) – for example, near the rear-side window – to measure the brightness of the space outside the rear-side window. In this case, the brightness can be determined based on the detection results of the illuminance sensor. Figure 3 The control unit 1110 controls the transmittance of the electronically controlled transmittance variable device 1620. Thus, even if the user 230 does not... Figure 3 The operation input unit 1107 can input operations, and the transmittance of the electronically controlled transmittance variable device 1620 can be adjusted according to the brightness of the space outside the rear side window, so that the recognizability of the spatial floating image 3 can be better maintained.

[0168] Furthermore, the example described above uses an electrically controlled variable transmittance device 1620, which is an example of a liquid crystal shutter. However, electronic paper can also be used as another example of an electrically controlled variable transmittance device 1620. Even when using electronic paper, the same effect as described above can be achieved. Moreover, the power consumption of electronic paper in maintaining intermediate color levels is very low. Therefore, compared with the case of using a liquid crystal shutter, a low-power spatial levitation image display device can be realized.

[0169] Figure 4K This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4K Spatial levitation image display device 1000 and Figure 4G The difference between this spatial levitation image display device and the previous one is that it has a transmissive self-emissive image display device 1650 instead of the transparent component 100. Other structures are the same as... Figure 4G The spatial levitation image display device has the same structure, so repeated descriptions are omitted.

[0170] exist Figure 4KIn the spatial levitation image display device 1000, after the image beam passes through the display surface of the transmissive self-emissive image display device 1650, a spatial levitation image 3 is formed outside the spatial levitation image display device 1000. That is, when an image is displayed using the transmissive self-emissive image display device 1650 as a two-dimensional flat panel display, the spatial levitation image 3 can be displayed as a floating image in front of the image on the transmissive self-emissive image display device 1650. At this time, the user 230 can simultaneously see two images with different depth positions. The transmissive self-emissive image display device 1650 can be constructed using existing technologies such as the transmissive organic EL panel disclosed in Japanese Patent Application Publication No. 2014-216761. In addition, the transmissive self-emissive image display device 1650 in Figure 3 Not shown in the image, but can be used as... Figure 3 It is a component of the spatial levitation image display device 1000 and is connected to other processing units such as the control unit 1110.

[0171] This allows for the display of both background and character objects on the transmissive self-emissive image display device 1650, followed by the movement of only the character objects to the suspended image 3 in front of the user. This provides the user 230 with a more effective "surprise effect" image experience.

[0172] Furthermore, if the interior of the spatial levitation image display device 1000 is in a light-blocking state, the background of the transmissive self-emissive image display device 1650 is sufficiently dark. Therefore, when the display device 1 does not display an image or the light source of the display device 1 does not emit light, and only the transmissive self-emissive image display device 1650 displays an image, for the user 230, the transmissive self-emissive image display device 1650 appears not to be a transmissive display but to be a typical two-dimensional flat panel display (in the embodiments of the present invention, the spatial levitation image 3 is displayed as a real optical image in a space where there is no screen, so if the light source of the display device 1 is not emitted, the predetermined display position of the spatial levitation image 3 is a space where nothing exists). Therefore, by using the transmissive self-emissive image display device 1650 to display images as if it were a typical two-dimensional flat panel display, and then suddenly displaying characters and objects in the air as spatial levitation images 3, a more effective "surprise effect" image experience can be provided to the user 230.

[0173] Furthermore, the darker the interior of the spatial levitation image display device 1000, the more the transmissive self-emissive image display device 1650 appears to be a two-dimensional flat panel display. Therefore, an absorptive polarizer (not shown) can be provided on one side of the spatial levitation image display device 1000 (the incident surface where the image light reflected from the polarization separation member 101B is incident on the transmissive self-emissive image display device 1650, i.e., the side of the transmissive self-emissive image display device 1650 opposite to the spatial levitation image 3) to transmit the polarization of the image light reflected from the polarization separation member 101B and absorb the polarization that is 90° out of phase with the polarization. In this way, the impact on the image light forming the spatial levitation image 3 is not significant, but the light incident from the outside through the transmissive self-emissive image display device 1650 into the interior of the spatial levitation image display device 1000 can be greatly reduced, making the interior of the spatial levitation image display device 1000 darker, which is preferable.

[0174] Figure 4L This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4L The space-floating image display device 1000 is Figure 4K A variation of the space-suspended image display device. The configuration orientation of the structure in the space-suspended image display device 1000 is... Figure 4K Unlike spatial levitation image display devices, it is different from... Figure 4F The configuration is similar to that of a spatial levitation image display device. Regarding the functions and operations of each structure, because... Figure 4K The spatial levitation image display device has the same structure, so repeated descriptions are omitted.

[0175] Figure 4L The same applies to the spatial levitation image display device. After the beam of image light passes through the transmissive self-emissive image display device 1650, a spatial levitation image 3 is formed on the side of the transmissive self-emissive image display device 1650 closer to the user 230.

[0176] Whether Figure 4K Examples of spatial levitation image display devices are still... Figure 4L In the example of the spatial levitation image display device, from the user's perspective (230), the spatial levitation image 3 is superimposed on the image of the transmissive self-emissive image display device 1650 (on the side in front of the user). Here, the position of the spatial levitation image 3 differs from the position of the image of the transmissive self-emissive image display device 1650 in the depth direction. Therefore, when the user moves their head (viewpoint), they can perceive the depth of the two images due to parallax. Thus, by displaying two images with different depth positions, a better naked-eye 3D image experience can be provided to the user without the need for stereoscopic glasses.

[0177] Figure 4M This diagram illustrates an example of the structure of a spatially suspended image display device. Figure 4M In the spatial levitation image display device 1000, relative to Figure 4G The polarization separation component 101B of the spatial levitation image display device has a second display device 1680 located on the depth side as viewed from the user. Other structures are similar to... Figure 4G The spatial levitation image display device has the same structure, so repeated descriptions are omitted.

[0178] Figure 4M In the illustrated structural example, the second display device 1680 is disposed at a depth of the display position of the spatial levitation image 3, with its image display surface facing the spatial levitation image 3. With this structure, from the user 230's perspective, the image displayed on the second display device 1680 and the spatial levitation image 3, both displayed at different depths, can be seen overlappingly. That is, it can be considered that the second display device 1680 is positioned in the direction of displaying the image facing the user 230 who is viewing the spatial levitation image 3. Furthermore, the second display device 1680... Figure 3 Not shown in the image, but can be used as... Figure 3 It is a component of the spatial levitation image display device 1000 and is connected to other processing units such as the control unit 1110.

[0179] in addition, Figure 4M The image light from the second display device 1680 of the spatial levitation image display device 1000 is seen by the user 230 after passing through the polarization separation member 101B. Therefore, in order to better transmit the image light from the second display device 1680 through the polarization separation member 101B, it is preferable that the image light output from the second display device 1680 is light with a polarization direction more suitable for transmission through the polarization separation member 101B. That is, the image light is preferably polarized light with the same polarization direction as the image light output from the display device 1. For example, if the image light output from the display device 1 is S-polarized light, it is preferable that the image light output from the second display device 1680 is also S-polarized light. Similarly, if the image light output from the display device 1 is P-polarized light, it is preferable that the image light output from the second display device 1680 is also P-polarized light.

[0180] Figure 4M The example of a spatial levitation image display device also displays a second image at a depth of the spatial levitation image 3, which in this respect has the same characteristics as... Figure 4K Examples of spatial levitation image display devices and Figure 4L The same effect as the example of a spatially suspended image display device. However, with Figure 4K Examples of spatial levitation image display devices and Figure 4LUnlike examples of spatial levitation image display devices, in Figure 4M In the example of the spatial levitation image display device, the beam of image light used to form the spatial levitation image 3 does not pass through the second display device 1680. Therefore, the second display device 1680 does not need to be a transmissive self-emissive image display device; it can be a liquid crystal display (LCD) as a two-dimensional flat panel display. The second display device 1680 can also be an organic EL display. Thus, with... Figure 4K Examples of spatial levitation image display devices and Figure 4L Compared to examples of spatially suspended image display devices. Figure 4M The example of a spatial levitation image display device enables the realization of the spatial levitation image display device 1000 at a lower cost.

[0181] Here, depending on the polarization distribution of the image light output from display device 1 and the performance of polarization separation component 101B, there is a possibility that a portion of the image light output from display device 1 may be reflected on polarization separation component 101B and go to second display device 1680. There is a possibility that this light (a portion of the image light) may be reflected again on the surface of second display device 1680 as stray light and be seen by the user.

[0182] Therefore, to prevent stray light, an absorptive polarizer can be provided on the surface of the second display device 1680. In this case, the absorptive polarizer can be an absorptive polarizer that transmits polarized image light output from the second display device 1680 and absorbs polarized light with a polarization 90° out of phase with the image light output from the second display device 1680. Furthermore, when the second display device 1680 is a liquid crystal display (LCD), an absorptive polarizer is also present on the image emission side inside the LCD. However, if the emission surface of the absorptive polarizer on the image emission side inside the LCD has a glass cover (a glass cover on the image display side), it will be impossible to prevent stray light generated by reflection of light from outside the LCD from the glass cover. Therefore, it is necessary to additionally provide the aforementioned absorptive polarizer on the surface of the glass cover.

[0183] Furthermore, when displaying images using the second display device 1680, which is a two-dimensional flat panel display, the spatially suspended image 3 can be displayed as an image on the side of the image on the second display device 1680 closer to the user (the side in front of the user). At this time, the user 230 can simultaneously see two images with different depth positions. By displaying the character using the spatially suspended image 3 and displaying the background on the second display device 1680, the user 230 can be provided with an effect as if they can see the space where the character is located in three dimensions.

[0184] Another effect can be achieved by displaying both the background and the character on the second display device 1680, and then moving only the character and the other objects to the space in front of the user to float the image 3. This can provide the user 230 with a more effective "surprise effect" image experience.

[0185] <Display Device>

[0186] Next, the display device 1 of this embodiment will be described using the accompanying drawings. The display device 1 of this embodiment includes an image display element 11 (liquid crystal display panel) and a light source device 13 constituting its light source. Figure 5 In the figure, the light source device 13 is shown in an unfolded perspective view along with the liquid crystal display panel.

[0187] The liquid crystal display panel (image display element 11) is as follows Figure 5 As indicated by arrow 30, an illumination beam with characteristics similar to laser light—namely, strong directional (straight-line travel) and a uniform polarization plane—is received from the light source device 13, which serves as a backlight device. The liquid crystal display panel (image display element 11) modulates the received illumination beam according to the input image signal. The modulated image light is reflected by the retroreflector 2 and passes through the transparent component 100 to form a spatially suspended image of a real image (see reference). Figure 1 ).

[0188] in addition, Figure 5 The display device 1 includes a liquid crystal display panel 11 constituting the display device 1, and a light direction conversion panel 54 that controls the pointing characteristics of the emitted light beam from the light source device 13. It also includes a narrow-angle diffuser (not shown) as needed. Specifically, polarizers are provided on both sides of the liquid crystal display panel 11, and image light with a specific polarization is emitted according to the intensity of the light modulated by the image signal (see reference). Figure 5 Arrow 30). Thus, the desired image, as highly directional (straight-line travel) polarized light, is projected onto the retroreflector 2 via the light direction conversion panel 54. After being reflected on the retroreflector 2, it is transmitted and reaches the viewer's eye outside the shop (space), forming a spatially suspended image 3. Alternatively, a protective cover 50 can be provided on the surface of the aforementioned light direction conversion panel 54 (see arrow 30). Figure 6 , Figure 7 ).

[0189] <Example 1 of a display device>

[0190] Figure 6 An example illustrating the specific structure of display device 1. Figure 6 exist Figure 5 The light source device 13 is equipped with a liquid crystal display panel 11 and a light direction conversion panel 54. The light source device 13... Figure 5The housing shown is made of materials such as plastic, and contains LED components 201 and a light guide 203. At the end face of the light guide 203, in order to... Figure 5 The lens shape shown converts the divergent light from each LED element 201 into a roughly parallel beam. It has a cross-sectional area that gradually increases towards the surface opposite the light-receiving part, and it allows for multiple total internal reflections during internal propagation, thereby gradually reducing the divergence angle. A liquid crystal display panel 11 constituting the display device 1 is mounted on the upper surface of the display device 1. Furthermore, an LED substrate 202 is mounted on one side (in this example, the left end face) of the housing of the light source device 13, on which LED (Light Emitting Diode) elements 201, serving as semiconductor light sources, and their control circuitry are mounted. A heat sink, used to cool the heat generated in the LED elements and control circuitry, can be mounted on the outer surface of the LED substrate 202.

[0191] Furthermore, on the frame (not shown) of the liquid crystal display panel mounted on the upper surface of the housing of the light source device 13, a liquid crystal display panel 11 mounted on the frame, and an FPC (Flexible Printed Circuits) (not shown) electrically connected to the liquid crystal display panel 11 are mounted. That is, the liquid crystal display panel 11, as an image display element, and the LED element 201, as a solid-state light source, are based on the control circuit constituting the electronic device (…). Figure 3 The image control unit 1160 modulates the intensity of the transmitted light using a control signal to generate a display image. At this time, the generated image light has a narrow diffusion angle and only a specific polarization component, thus approximating a surface-emitting laser image source driven by an image signal, resulting in a novel image display device unlike any previously known. Furthermore, currently, obtaining a laser beam of the same size as the image obtained by the aforementioned display device 1 using a laser device is technically and safety-wise impossible. Therefore, in this embodiment, for example, a beam emitted by a conventional light source having LED elements is used to obtain light that approximates the surface-emitting laser image light.

[0192] Next, the structure of the optical system housed within the housing of the light source device 13 will be described with reference to... Figure 6 as well as Figure 7 A detailed explanation will follow.

[0193] because Figure 6 and Figure 7 This is a cross-sectional view, so only one of the multiple LED elements 201 constituting the light source is shown. They are transformed into approximately collimated light by the shape of the light-receiving end face 203a of the light guide 203. Therefore, the light-receiving part of the light guide end face is mounted in a predetermined positional relationship with the LED element.

[0194] Furthermore, the light guide 203 is formed, for example, using a light-transmitting resin such as acrylic resin. The LED light-receiving surface at the end of the light guide 203 has, for example, an outer peripheral surface in the shape of a convex cone obtained by rotating a parabolic section. Its top has a recess, and a convex portion (i.e., a convex lens surface) is formed in the center of this recess. Furthermore, a convex lens surface protruding outwards (or a concave lens surface recessed inwards) is formed in the center of its flat portion (not shown). Additionally, the light-receiving portion of the light guide on which the LED element 201 is mounted has a parabolic shape forming a conical outer peripheral surface, and is set within an angle range that allows light emitted from the LED element in the peripheral direction to be totally internally reflected, or a reflective surface is formed.

[0195] On the other hand, LED elements 201 are respectively disposed at predetermined positions on the surface of its circuit board, i.e., LED substrate 202. The LED substrate 202 is disposed and fixed relative to the LED collimator (light-receiving end face 203a) such that the LED elements 201 on its surface are respectively located at the center of the aforementioned recess.

[0196] According to this structure, the shape of the light-receiving end face 203a of the light guide 203 can be used to make the light emitted from the LED element 201 become approximately parallel light and output, thereby improving the utilization efficiency of the generated light.

[0197] As described above, the light source device 13 is constructed by mounting a light source unit consisting of a plurality of LED elements 201 arranged as light sources at the light-receiving end face 203a, which is provided on the end face of the light guide 203. For the diverging beam from the LED elements 201, the light-receiving end face 203a of the light guide end face makes it approximately parallel to the light, and guides the light inside the light guide 203 (in a direction parallel to the paper surface) as shown by the arrow. The beam direction conversion unit 204 directs the light to be emitted towards the liquid crystal display panel 11 (in a direction perpendicular to the paper surface) which is arranged approximately parallel to the light guide 203. By optimizing the distribution (density) of the beam direction conversion unit 204 with the shape of the inside or surface of the light guide, the uniformity of the beam incident on the liquid crystal display panel 11 can be controlled.

[0198] The aforementioned beam direction conversion unit 204, by utilizing the shape of the light guide surface or by providing portions with different refractive indices within the light guide, directs the beam propagating within the light guide towards the liquid crystal display panel 11 (in a direction perpendicular to the paper surface) arranged approximately parallel to the light guide 203. At this time, for the liquid crystal display panel 11, with the viewpoint positioned directly opposite the center of the screen and the diagonal dimension of the screen aligned with the center, the brightness of the center and periphery of the screen is compared. A relative brightness ratio of 20% or higher is sufficient for practical use, and a ratio exceeding 30% indicates even better performance.

[0199] in addition, Figure 6 This is a cross-sectional view illustrating the structure and function of the light source in the light source device 13, which includes the light guide 203 and the LED element 201, according to this embodiment. Figure 6 In this light source device 13, for example, there is a light guide 203 made of plastic or the like with a beam direction conversion unit 204 provided on its surface or inside, an LED element 201 as a light source, a reflector 205, a phase difference plate 206, a cylindrical lens, etc., and a liquid crystal display panel 11 with polarizers on the light source incident surface and the image light exit surface is mounted on its upper surface.

[0200] Furthermore, a thin film or sheet-like reflective polarizer 49 is provided on the light incident surface (lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, so that one polarization (e.g., P-light) 212 of the natural light beam 210 emitted from the LED element 201 is selectively reflected. The reflected light is reflected again by a reflective sheet 205 provided on one surface (lower surface in the figure) of the light guide 203 and goes to the liquid crystal display panel 11. Therefore, a phase retardation plate (λ / 4 waveplate) is provided between the reflective sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizer 49, so that the light is reflected on the reflective sheet 205 and passes through the phase retardation plate twice, thereby changing the reflected beam from P-polarization to S-polarization, improving the utilization efficiency of the light source light as image light. The light intensity is modulated by the liquid crystal display panel 11 according to the image signal of the image beam ( Figure 6 Arrow 213) is incident on retroreflector 2. After reflection on retroreflector 2, a spatial levitation image of real image can be obtained.

[0201] Figure 7 Is with Figure 6 Similarly, a cross-sectional view is used to illustrate the structure and function of the light source in this embodiment, which performs polarization transformation in the light source device 13 including the light guide 203 and the LED element 201. The light source device 13 also similarly includes, for example, a light guide 203 made of plastic or the like with a beam direction transformation unit 204 provided on its surface or internally, an LED element 201 serving as the light source, a reflector 205, a phase retardation plate 206, a cylindrical lens, etc. A liquid crystal display panel 11, having polarizers on the light source incident surface and the image light exit surface, is mounted as an image display element on the upper surface of the light source device 13.

[0202] A thin film or sheet-like reflective polarizer 49 is provided on the light incident surface (lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, so that a certain polarization (e.g., S-ray) 211 in the natural light beam 210 emitted from the LED element 201 is selectively reflected. That is, Figure 7 In the example, the selective reflection characteristics of the reflective polarizer 49 are similar to... Figure 7The reflected light is reflected by a reflective sheet 205 located on one surface of the light guide 203 (lower in the figure) and then travels back to the liquid crystal display panel 11. A phase retardation plate (λ / 4 waveplate) is provided between the reflective sheet 205 and the light guide 203, or between the light guide 203 and the reflective polarizer 49. The light is then reflected by the reflective sheet 205 and passes through the phase retardation plate twice, thereby changing the reflected beam from S-polarization to P-polarization and improving the utilization efficiency of the light source as image light. The light intensity is modulated by the liquid crystal display panel 11 according to the image signal of the image beam (…). Figure 7 Arrow 214) is incident on the retroreflector 2. After reflection on the retroreflector 2, a spatially suspended image of a real image can be obtained.

[0203] exist Figure 6 and Figure 7 In the light source device shown, besides the function of the polarizer on the light incident surface of the corresponding liquid crystal display panel 11, the theoretically achievable contrast ratio is the product of the reciprocal of the orthogonal transmittance of the reflective polarizer and the reciprocal of the orthogonal transmittance obtained from the two polarizers attached to the liquid crystal display panel, since a polarization component is reflected by a reflective polarizer. Therefore, a high contrast ratio can be obtained. In fact, experiments have confirmed that the contrast ratio of the displayed image is improved by more than 10 times. As a result, high-quality images comparable to those obtained with self-emissive organic EL displays can be obtained.

[0204] <Example 2 of a display device>

[0205] Figure 8 This is another example of the specific structure of the display device 1. The light source device 13 is constructed by housing LEDs, collimators, composite diffusers, light guides, etc., in a housing such as plastic, and a liquid crystal display panel 11 is mounted on its upper surface. In addition, an LED substrate is mounted on one side of the housing of the light source device 13, on which LED (Light Emitting Diode) elements 14a and 14b, which serve as semiconductor light sources, and their control circuits are mounted. A heat sink 103, which is used to cool the heat generated in the LED elements and the control circuits, is mounted on the outer side of the LED substrate.

[0206] Furthermore, a liquid crystal display panel 11 mounted on the upper surface of the housing, and an FPC (Flexible Printed Circuits) 403 electrically connected to the liquid crystal display panel 11 are provided on the liquid crystal display panel frame. That is, the liquid crystal display panel 11, as a liquid crystal display element, and the LED elements 14a and 14b, as solid-state light sources, together modulate the intensity of transmitted light based on control signals from the control circuit (not shown here) constituting the electronic device to generate a display image.

[0207] <Example 3 of a display device>

[0208] Next, use Figure 9 Another example illustrating the specific structure of display device 1 (Example 3 of the display device). The light source device of this display device 1 uses a collimator 18 to transform the divergent beam of light (a mixture of P-polarized and S-polarized light) from the LED into a substantially parallel beam, and uses the reflective surface of the reflective light guide 304 to reflect it towards the liquid crystal display panel 11. The reflected light is incident on a reflective polarizer 49 disposed between the liquid crystal display panel 11 and the reflective light guide 304. The reflective polarizer 49 allows light of a specific polarization (e.g., P-polarized light) to pass through, causing the transmitted polarized light to enter the liquid crystal display panel 11. Here, other polarizations (e.g., S-polarized light) are reflected by the reflective polarizer 49 and return to the reflective light guide 304.

[0209] The reflective polarizer 49 is tilted relative to the liquid crystal display panel 11 in a manner that is not perpendicular to the principal ray of light from the reflective surface of the reflective light guide 304. The principal ray of light reflected from the reflective polarizer 49 is incident on the transmissive surface of the reflective light guide 304. Light incident on the transmissive surface of the reflective light guide 304 passes through the back surface of the reflective light guide 304 and is reflected on the reflective plate 271 by the λ / 4 waveplate 270, which serves as a phase retardation plate. The light reflected on the reflective plate 271 passes through the λ / 4 waveplate 270 again and passes through the transmissive surface of the reflective light guide 304. The light passing through the transmissive surface of the reflective light guide 304 is then incident on the reflective polarizer 49 again.

[0210] At this point, the light that is incident on the reflective polarizer 49 again passes through the λ / 4 waveplate 270 twice, so its polarization is transformed into a polarization that can pass through the reflective polarizer 49 (e.g., P-polarization). Thus, the polarization-transformed light passes through the reflective polarizer 49 and enters the liquid crystal display panel 11. Alternatively, the polarization design in the polarization transformation can be reversed compared to the above description (switching S-polarization and P-polarization).

[0211] As a result, the light from the LEDs is uniformly polarized (e.g., P-polarized) and incident on the liquid crystal display panel 11, where brightness modulation is performed accordingly based on the image signal to display an image on the panel surface. Multiple LEDs constituting the light source are shown in the same way as in the example above (but because it is a longitudinal cross-sectional view, ...). Figure 9 (Only one is shown in the figure), and they are installed in the specified positions relative to the collimator 18.

[0212] Furthermore, the collimator 18 is formed, for example, using a light-transmitting resin such as acrylic resin or glass. The collimator 18 may have an outer peripheral surface with a convex conical shape obtained by rotating a parabolic section. Additionally, at the center of the top portion of the collimator 18 (the side opposite to the LED substrate 102), a recess with a protrusion (i.e., a convex lens surface) may be formed. Furthermore, at the center of the planar portion of the collimator 18 (the side opposite to the aforementioned top), a convex lens surface protruding outwards (or it may be a concave lens surface recessed inwards) may be formed. Additionally, the parabolic surface forming the conical outer peripheral surface of the collimator 18 is set within an angle range that allows light emitted from the LED in the peripheral direction to undergo total internal reflection within it, or it forms a reflective surface.

[0213] In addition, the LEDs are respectively disposed at predetermined positions on the surface of its circuit board, i.e., the LED substrate 102. The LED substrate 102 is configured and fixed relative to the collimator 18 such that the LEDs on its surface are respectively located at the center of the top of the convex conical shape (or the recess if the top has a recess).

[0214] According to this structure, under the action of the collimator 18, the light emitted from the LED, especially the light emitted from its central portion, is converged into parallel light by the convex lens surface forming the shape of the collimator 18. Furthermore, light emitted from other portions towards the periphery is reflected by the parabolic surface of the conical outer peripheral surface of the collimator 18, and similarly converged into parallel light. In other words, by using the collimator 18, which has a convex lens in its central portion and a parabolic surface in its peripheral portion, almost all the light generated by the LED can be output as parallel light, thereby improving the utilization efficiency of the generated light.

[0215] and then, Figure 9The light, transformed into approximately parallel light by collimator 18, is reflected by reflective light guide 304. A portion of this light, with a specific polarization, passes through reflective polarizer 49, and another portion, reflected by reflective polarizer 49, passes through light guide 304 again. This light is reflected by reflective plate 271, located opposite to the liquid crystal display panel 11 relative to reflective light guide 304. Here, the light undergoes polarization transformation twice by passing through λ / 4 waveplate 270, which acts as a phase difference plate. The light reflected by reflective plate 271 passes through light guide 304 again and is incident on reflective polarizer 49, which is positioned on the opposite side. Because the incident light has been polarized, it can pass through reflective polarizer 49, resulting in a uniform polarization direction incident on the liquid crystal display panel 11. As a result, all light from the light source can be utilized, achieving a 2-fold increase in geometrical optics efficiency. Furthermore, the polarization degree (extinction ratio) of the reflective polarizer is multiplied by the overall extinction ratio of the system, thus significantly improving the overall contrast of the display device by using the light source device of this embodiment. Additionally, by adjusting the surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflective plate 271, the reflection diffusion angle of light on each reflective surface can be adjusted. To improve the uniformity of light incident on the liquid crystal display panel 11, the surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflective plate 271 can be adjusted according to each design.

[0216] in addition, Figure 9 For polarized light incident perpendicular to the λ / 4 waveplate 270, the phase difference does not need to be λ / 4. Figure 9 In this structure, any phase difference plate that allows polarized light to pass through twice, thus changing its phase by 90° (λ / 2), is sufficient. The thickness of the phase difference plate can be adjusted accordingly based on the incident angle distribution of the polarized light.

[0217] <Example 4 of a display device>

[0218] Furthermore, using Figure 10 Another example illustrating the structure of the optical system, such as the light source device, of the display device is given (Example 4 of the display device). Example 4 of the display device is a structural example in which a diffuser is used instead of the reflective light guide 304 in the light source device of Example 3 of the display device. Specifically, on the light emitting side of the collimator 18, two optical sheets (optical sheet 207A and optical sheet 207B) are used to transform the diffusion characteristics in the vertical and horizontal directions (front and back directions of the figure, not shown), so that the light from the collimator 18 is incident between the two optical sheets (diffusers).

[0219] Alternatively, the aforementioned optical sheet can be replaced with a single sheet instead of a two-sheet structure. With a single-sheet structure, the vertical and horizontal diffusion characteristics are adjusted by the fine shapes of the front and back sides of the single optical sheet. Alternatively, multiple diffuser sheets can be used to distribute the effect. Here, in Figure 10 In the example, for the reflection diffusion characteristics determined by the front and back shapes of optical sheets 207A and 207B, the number of LEDs, the divergence angle of the LED substrate (optical element) 102, and the optical specifications of the collimator 18 are optimized as design parameters in a way that makes the surface density of the light beam emitted from the liquid crystal display panel 11 uniform. That is, the diffusion characteristics are adjusted by using the surface shapes of multiple diffusers instead of a light guide.

[0220] Figure 10 In this example, the polarization transformation is performed using the same method as in Example 3 of the aforementioned display device. That is, in Figure 10 In this example, the reflective polarizer 49 can be configured to reflect S-polarized light (transmit P-polarized light). In this case, P-polarized light emitted from the light source, i.e., the LED, is transmitted, and the transmitted light enters the liquid crystal display panel 11. S-polarized light emitted from the light source, i.e., the LED, is reflected, and the reflected light passes through... Figure 10 The phase retardation plate 270 is shown. Light passing through the phase retardation plate 270 is reflected by the reflector plate 271. The light reflected by the reflector plate 271 passes through the phase retardation plate 270 again and is converted into P-polarized light. The polarized light passes through the reflective polarizer 49 and is incident on the liquid crystal display panel 11.

[0221] in addition, Figure 10 For polarized light incident perpendicular to the λ / 4 waveplate 270, the phase difference does not need to be λ / 4. Figure 10 In this structure, any phase retardation plate that allows polarized light to pass through twice, thus changing its phase by 90° (λ / 2), is sufficient. The thickness of the phase retardation plate can be adjusted accordingly based on the incident angle distribution of the polarized light. Furthermore, Figure 10 Similarly, in polarization transformation, the polarization design can be reversed compared to the above description (swapping the S polarization and P polarization).

[0222] Regarding the emitted light from the liquid crystal display panel 11, in typical TV-use devices, in the horizontal direction of the screen (using... Figure 12 (a) represented by the X-axis) and the vertical direction of the image (using) Figure 12 (b) representing the Y-axis all exhibit the same diffusion characteristics. In contrast, the diffusion characteristics of the emitted light beam from the liquid crystal display panel of this embodiment are as follows: Figure 12As shown in Example 1, the viewing angle is 13 degrees when the brightness is 50% of that of a front-viewing device (0 degrees), which is about 1 / 5 of the 62 degrees of a typical TV device. Similarly, for the vertical viewing angle, the reflection angle of the reflective light guide and the area of ​​the reflective surface are optimized to make it uneven vertically, suppressing the upper viewing angle to about 1 / 3 of the lower viewing angle. As a result, compared with existing LCD TVs, the amount of image light heading towards the viewing direction is significantly increased, with brightness more than 50 times.

[0223] Furthermore, if adopted Figure 12 The viewing angle characteristics shown in Example 2 indicate that the viewing angle is 5 degrees when the brightness is 50% of that of a front-viewing device (0 degrees), which is 1 / 12 of the 62 degrees of a typical TV device. Similarly, for the vertical viewing angle, the reflection angle of the reflective light guide and the area of ​​the reflective surface are optimized to make it uniform vertically and to suppress the viewing angle to about 1 / 12 of that of a typical TV device. As a result, the amount of image light heading towards the viewing direction is significantly increased compared to existing LCD TVs, with brightness more than 100 times greater.

[0224] By narrowing the viewing angle as described above, the amount of light beam heading in the viewing direction can be concentrated, thus significantly improving light utilization efficiency. As a result, even when using a typical LCD panel for TV applications, a significant increase in brightness can be achieved with the same power consumption by controlling the light diffusion characteristics of the light source device, enabling an image display device compatible with information display systems facing bright outdoor environments.

[0225] When using a large LCD panel, the light from the periphery of the screen is directed inwards so that it reaches the viewer when the viewer is facing the center of the screen, thereby improving the overall brightness of the screen. Figure 11 The convergence angles of the long and short sides of the panel were determined using the distance L between the viewer and the panel, and the panel size (16:10 aspect ratio) as parameters. In portrait mode, the convergence angle can be set accordingly for the short side. For example, with a 22″ panel used in portrait mode at a viewing distance of 0.8m, setting the convergence angle to 10 degrees will allow image light from the four corners of the screen to effectively reach the viewer.

[0226] Similarly, when viewing a 15″ panel in portrait mode at a viewing distance of 0.8m, setting the convergence angle to 7 degrees allows image light from the four corners of the screen to effectively reach the viewer. As described above, by directing image light from the periphery of the screen to the viewer in the most suitable position for viewing the center of the screen, based on the size of the LCD panel and whether it is used in portrait or landscape mode, the overall brightness of the screen can be improved.

[0227] As the basic structure, as described above Figure 9 The light source device is used to direct a beam of light with narrow-angle pointing characteristics onto the liquid crystal display panel 11. The brightness is modulated accordingly according to the image signal, so that the image information displayed on the screen of the liquid crystal display panel 11 is reflected on the retroreflector to obtain a spatial levitation image, which is then displayed outdoors or indoors via the transparent component 100.

[0228] Using the display device and light source device of one embodiment of the present invention described above, a spatial levitation image display device with higher light utilization efficiency can be realized.

[0229] <Examples of image display processing in spatial levitation image display devices>

[0230] Next, for one example of the technical problem to be solved by the image processing in this embodiment, using... Figure 13A The following explanation is provided. In the spatial levitation image display device 1000, from the user's perspective, the deep side of the spatial levitation image 3 is located inside the housing of the spatial levitation image display device 1000, and in sufficiently dark conditions, the user sees the background of the spatial levitation image 3 as black.

[0231] Here, use Figure 13A This section explains an example of displaying the character "Panda" 1525 within a spatial levitation image 3. First, Figure 3 The image control unit 1160 for such Figure 13A The image shown in (1) includes a pixel region depicting the character “panda” 1525 and a transparent information region 1520 serving as a background image, and the pixel region depicting the character “panda” 1525 and the transparent information region 1520 serving as a background image are distinguished.

[0232] Regarding the method for distinguishing and recognizing character images from background images, for example, the image processing of the image control unit 1160 can be configured to process the background image layer and the character image layer located in front of the background image layer as different layers, and to distinguish and recognize the character image and the background image based on the overlap relationship when these layers are composited.

[0233] Here, the image control unit 1160 identifies the black pixels of the pixels depicting objects such as character images as different from the transparent information pixels. However, the pixel brightness of both the black pixels depicting objects and the transparent information pixels is 0. In this case, when displaying the spatial levitation image 3, the black pixels in the image depicting the character "panda" 1525 have no brightness difference from the pixels in the transparent information area 1520, which serves as the background image. Therefore, in the spatial levitation image 3, as... Figure 13AAs shown in (2), the black pixels in the image depicting the character "Panda" 1525 and the pixels in the transparent information area 1520 have no brightness and are perceived by the user as optically identical black spaces. That is, the black parts in the image depicting the object, namely the character "Panda" 1525, blend into the background, and only the non-black parts in the character "Panda" 1525 can be recognized as images floating in the display area of ​​the spatial floating image 3.

[0234] For one example of image processing in this embodiment, using Figure 13B Please provide an explanation. Figure 13B This indicates that it can better solve the problem. Figure 13A This image illustrates an example of image processing addressing the technical problem of black areas of the object blending seamlessly into the background. Figure 13B In (1) and (2), the display state of the spatial levitation image 3 is shown on the upper side, and the input / output characteristics of the image processing of the object's image are shown on the lower side. Additionally, the image of the object (character "Panda" 1525) and / or its corresponding data can be obtained from... Figure 3 The data can be read from the storage unit 1170 or the memory 1109. Alternatively, the data can be input from the image signal input unit 1131. Alternatively, the data can be obtained via the communication unit 1132.

[0235] Here, in Figure 13B In state (1), the input-output characteristics of the object's image processing are linear without any special adjustment. In this case, it is related to... Figure 13A In (2) the same display state, the black image area of ​​the object blends into the background. In contrast, in Figure 13B In (2), the image control unit 1160 of this embodiment adjusts the input-output characteristics of the image processing of the object (character "panda" 1525) to the input-output characteristics shown below.

[0236] That is, the image control unit 1160 performs image processing on the image of the object (character "Panda" 1525) with the following input-output characteristics: for each pixel of the input image, the brightness value of the pixel in the low-brightness area is increased to obtain an output pixel. After the image processing with these input-output characteristics is performed on the image of the object (character "Panda" 1525), the image including the image of the object (character "Panda" 1525) is input to the display device 1 for display. Thus, the display state of the spatial levitation image 3 is as follows: Figure 13BAs shown in the upper part of (2), the brightness of the black pixel area in the image depicting the character "Panda" 1525 is increased. As a result, the black areas in the image depicting the character "Panda" 1525 are not blended into the black background, allowing users to distinguish them and better display the object.

[0237] That is, by using Figure 13B The image processing of (2) displays an area of ​​the image of the object, namely the character "panda" 1525, which can be distinguished from the black background inside the housing of the spatial levitation image display device 1000 seen through the window, thus improving the recognizability of the object. Therefore, even if the object was before the aforementioned image processing (i.e., from...) Figure 3 When the image of the object and / or its corresponding data is read from the storage unit 1170 or the memory 1109, or when the image of the object is input from the image signal input unit 1131, or when the data of the object is obtained via the communication unit 1132, etc., an object whose pixels constituting the object include pixels with a brightness value of 0 can also be transformed into an object with increased brightness value of pixels in a low brightness area through image processing of the input-output characteristics performed by the image control unit 1160, and then displayed on the display device 1, and transformed into a spatial levitation image 3 by the optical system of the spatial levitation image display device 1000.

[0238] That is, the pixels of the object after image processing that constitutes the input and output characteristics do not contain pixels with a brightness value of 0. After being transformed into such a state, it is displayed on the display device 1 and transformed into a spatial levitation image 3 by the optical system of the spatial levitation image display device 1000.

[0239] In addition, Figure 13B In the image processing of (2), it is implemented only on a region of the image of the object (character "Panda" 1525). Figure 13B The image processing method with input-output characteristics of (2) is configured, for example, in the image processing of the image control unit 1160, to process the background image layer and the character image layer located in front of the background image layer as different layers, and to perform the processing on the character image layer. Figure 13B Image processing of the input and output characteristics of (2) does not apply to the background image layer.

[0240] These layers are then composited, resulting in, as shown Figure 13B As shown in (2), only the character image is subjected to image processing that brightens the low-brightness areas of the input image. Alternatively, as another method, it can be configured such that after compositing the character image layer with the background image layer, only the area of ​​the character image is processed. Figure 13B Image processing with input-output characteristics of (2).

[0241] Furthermore, in image processing that brightens low-brightness areas in the input-output characteristics of an input image, the input-output image characteristics used are not limited to... Figure 13B Example (2). Any image processing that can brighten low brightness is acceptable, such as so-called brightness adjustment. Alternatively, image processing as disclosed in International Publication No. 2014 / 162533 can be performed to improve recognizability by controlling the gain used to change the weights of the Retinex processing.

[0242] Based on the above explanation Figure 13B (2) Image processing can make black areas in the area of ​​an image that depicts characters and objects not blend into the black background, so that users can recognize them and achieve better display.

[0243] In addition, Figure 13A , Figure 13B In the example, a spatially suspended image display device with a seemingly black background (e.g.) Figures 4A-4G Spatial levitation image display device 1000 Figure 4I , Figure 4J Taking a spatial levitation image display device 1000 (with the rear side window shaded) as an example, its technical problems and improved image processing have been explained. However, this image processing is also effective in devices other than these spatial levitation image display devices.

[0244] Specifically, in Figure 4H Space levitation image display device 1000 and Figure 4I , Figure 4J In the spatial levitation image display device 1000 with the rear side window unobstructed, the background of the spatial levitation image 3 is not black, but rather the scenery behind the spatial levitation image display device 1000 through the window. In this case, Figure 13A and Figure 13B The technical issues described also exist.

[0245] That is, the black parts of the image depicting the object, namely the character "panda" 1525, blend into the scenery behind the space-suspended image display device 1000, which is separated by a window. The same applies in this case, through the use of... Figure 13B The image processing of (2) enables the black parts in the image depicting the object, namely the character "panda" 1525, to be distinguishably identified from the scenery behind the space-suspended image display device 1000 through the window, thereby improving the recognizability of the object.

[0246] That is, by using Figure 13BThe image processing of (2) shows that the area of ​​the image of the object, namely the character "panda" 1525, can be distinguished from the scenery behind the space-suspended image display device 1000 through the window, and the object, namely the character "panda" 1525, is better identified in front of the scenery, thus improving the recognizability of the object.

[0247] In addition, Figure 4K , Figure 4L , Figure 4M In the spatial levitation image display device 1000, as described above, when other images (such as images from the transmissive self-emissive image display device 1650 or the second display device 1680) are displayed at a depth position different from that of the spatial levitation image 3, the background of the spatial levitation image 3 is not black but rather that other image. In this case, Figure 13A and Figure 13B The technical issues described also exist.

[0248] That is, the black portion of the image depicting the object, namely the character "panda" 1525, blends seamlessly with the other images shown at a depth position different from that of the spatially suspended image 3. The same applies in this case, through the use of... Figure 13B The image processing of (2) enables the black parts in the image depicting the object, namely the character “panda” 1525, to be distinguishably identified from the other images mentioned above, thus improving the recognizability of the object.

[0249] That is, by using Figure 13B The image processing of (2) shows that the area of ​​the image of the object, namely the character "panda" 1525, can be distinguished from the other images mentioned above, and the object, namely the character "panda" 1525, can be better identified as being in front of the other images mentioned above, thus improving the recognizability of the object.

[0250] For one example of image display processing in this embodiment, using Figure 13C Please provide an explanation. Figure 13C This is an example of image display in this embodiment, showing a display of both the spatially suspended image 3 and other images, namely the second image 2050. The second image 2050 can correspond to... Figure 4K or Figure 4L The image displayed by the transmissive self-emissive image display device 1650. Additionally, the second image 2050 can also correspond to... Figure 4M The second display device 1680 displays images.

[0251] Right now, Figure 13C The image shows an example of Figure 4K , Figure 4L , Figure 4MA specific example of image display by the spatial levitation image display device 1000. In this example, a bear character is shown in spatial levitation image 3. The area outside the bear character in spatial levitation image 3 is black, as spatial levitation images are transparent. Additionally, the second image 2050 is a background image depicting a plain, mountains, and the sun.

[0252] Here, Figure 13C In the image, the spatial levitation image 3 and the second image 2050 are displayed at different depth positions. The user 230 views the two images, spatial levitation image 3 and the second image 2050, in the line of sight of arrow 2040, thus allowing the user 230 to view the images in an overlapping state. Specifically, the bear character in spatial levitation image 3 can be seen overlapping in front of the background of plains, mountains, and sun depicted in the second image 2050.

[0253] Here, the spatially suspended image 3 is projected as a real image in the air, so when the user 230 slightly moves their viewpoint, they can discern the distance between the spatially suspended image 3 and the second image 2050 due to parallax. Therefore, the user 230 can view the two images in an overlapping state, while simultaneously experiencing a stronger sense of spatial suspension from the spatially suspended image 3.

[0254] For one example of image display processing in this embodiment, using Figure 13D Please provide an explanation. Figure 13D (1) is Figure 13C In this example of image display, the image of the spatial levitation image 3 is viewed from the direction of the user 230's gaze. Here, a bear character is displayed in the spatial levitation image 3. The area outside the bear character in the spatial levitation image 3 is black, as the spatial levitation image is transparent.

[0255] Figure 13D (2) is Figure 13C In this example of image display, the second image 2050 is viewed from the direction of the user 230's line of sight. In this example, the second image 2050 is a background image depicting a plain, mountains, and the sun.

[0256] Figure 13D (3) represents Figure 13C In the example of image display in this embodiment, a diagram showing the state of the second image 2050 and the spatial levitation image 3 being seen overlaid in the direction of the user 230's gaze. Specifically, the bear character of the spatial levitation image 3 can be seen overlaid in front of the background of the plains, mountains, and sun depicted in the second image 2050.

[0257] Here, when both the spatial levitation image 3 and the second image 2050 are displayed simultaneously, it is preferable to maintain a balance in the brightness of both images to better ensure the recognizability of the spatial levitation image 3. If the second image 2050 is too bright compared to the brightness of the spatial levitation image 3, the displayed image of the spatial levitation image 3 will become transparent, allowing the background, i.e., the second image 2050, to be seen clearly and transparently.

[0258] Therefore, the output of the light source of the spatial levitation image 3 and the display image brightness of the display device 1, the output of the light source of the display device displaying the second image 2050 and the display image brightness of the display device can be set such that the brightness per unit area of ​​the spatial levitation image 3 at the display position of the spatial levitation image 3 is greater than the brightness per unit area of ​​the image light from the second image 2050 to the display position of the spatial levitation image 3.

[0259] Furthermore, as long as this condition is met when both the spatial floating image 3 and the second image 2050 are displayed simultaneously, control can be implemented to reduce the brightness of the second image 2050 by decreasing the output of the light source of the display device displaying the second image 2050 and / or the brightness of the displayed image on the display device. This control can be achieved through... Figure 3 The control unit 1110 controls the display device 1 and the display device that displays the second image 2050. Figure 4K or Figure 4L Transmissive self-emissive image display device 1650 or Figure 4M This is achieved through a second display device 1680.

[0260] Furthermore, during the switching from the first display mode to the second display mode, if it is necessary to reduce the brightness of the second image 2050, the brightness can be reduced uniformly across the entire screen of the second image 2050. Alternatively, instead of uniformly reducing the brightness across the entire screen of the second image 2050, the portion of the spatial levitation image 3 where the object to be displayed is made to have the highest brightness reduction effect, with the brightness reduction effect gradually reduced around its perimeter. This is because reducing the brightness of the second image 2050 only for the overlapping portion of the spatial levitation image 3 within the second image 2050 is sufficient to ensure the recognizability of the spatial levitation image 3.

[0261] Here, the spatial levitation image 3 and the second image 2050 are displayed at different depth positions, so when the user 230 slightly changes their viewpoint, the overlapping position of the spatial levitation image 3 on the second image 2050 will change due to parallax. Therefore, in the switching from the first display mode to the second display mode, when the brightness of the entire screen of the second image 2050 is reduced unevenly, it is not preferable to sharply reduce the brightness based on the outline of the object displayed in the spatial levitation image 3. Instead, it is preferable to gradually change the brightness reduction effect according to the position as described above, and perform a gradual processing of the brightness reduction effect.

[0262] Furthermore, if the position of the object displayed in the spatial levitation image 3 is approximately the center of the spatial levitation image 3, then in the spatial levitation image display device 1000, the position with the highest brightness reduction effect in the gradient processing of the brightness reduction effect can be set to the center of the spatial levitation image 3.

[0263] Based on the image display processing of this embodiment described above, user 230 is able to better view the spatial levitation image 3 and the second image 2050.

[0264] Alternatively, it can be controlled so that the second image 2050 is not displayed when the spatial levitation image 3 is displayed. When the second image 2050 is not displayed, the spatial levitation image 3 is more recognizable, so it is suitable for applications such as the spatial levitation image display device 1000 where the user must reliably see the spatial levitation image 3 when it is displayed.

[0265] <Example 2>

[0266] As Embodiment 2 of the present invention, another example of the structure of the spatial levitation image display device will be described. Furthermore, the optical system incorporated in the spatial levitation image display device described in Embodiment 1 is modified in this embodiment. Figure 14 (1) or Figure 14 The optical system shown in (2) is described in this embodiment. The differences from Embodiment 1 are explained in this embodiment, and repeated descriptions of the same structure as in Embodiment 1 are omitted. In addition, in the following description of this embodiment, it is specified that polarization (polarized light) and another polarization (polarized light) are polarizations (polarized light) that are 90° out of phase.

[0267] Figure 14 (1) is an example of the optical system and optical path of this embodiment. Figure 14 The optical system shown in (1) is in Figure 2C In the optical system, the display device 1 is moved closer to the polarization separation component 101B, making the overall optical system more compact. For Figure 14 (1) is marked with Figure 2CThe same reference numerals are used in the accompanying drawings, and repeated detailed descriptions are omitted.

[0268] Figure 14 In (1), with Figure 2C Similarly, image light with a predetermined polarization (P-polarization in the figure) emitted from display device 1 travels vertically from the image display surface of display device 1. Here, the polarization separation component 101B and... Figure 2C Similarly, a predetermined polarized light (P-polarized light in the figure) emitted from the display device 1 is selectively transmitted, while another polarized light (S-polarized light in the figure) is reflected.

[0269] Thus, the image light of a predetermined polarization (P-polarized in the figure) traveling vertically from the image display surface of the display device 1 passes through the polarization separating member 101B and reaches the retroreflector 2 to which the λ / 4 waveplate 21 is attached. The image light, retroreflected on the retroreflector 2 and traveling again towards the polarization separating member 101B, passes through the λ / 4 waveplate 21 twice, thereby changing from the predetermined polarization (P-polarized in the figure) emitted from the display device 1 to another polarization (S-polarized in the figure). The image light traveling again towards the polarization separating member 101B is now of another polarization (S-polarized in the figure), and is therefore reflected on the polarization separating member 101B towards the position where the user should be located. The direction of travel of the image reflected on the polarization separating member 101B is determined based on the configuration angle of the polarization separating member 101B.

[0270] Figure 14 In example (1), the image light traveling toward the polarization separation component 101B is reflected at a right angle onto the polarization separation component 101B and travels as shown in the figure. The image light reflected onto the polarization separation component 101B forms a spatial levitation image 3A. The spatial levitation image 3A can be viewed well by the user from the direction of arrow A.

[0271] Here, regarding the retroreflective characteristics of the retroreflector 2, there exists a relationship where the optical path length of the image light emitted from the display device 1 to the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 to the formation position of the spatially suspended image 3A. The formation position of the spatially suspended image 3A in the direction of travel of the image light reflected from the polarization separation component 101B is determined by this relationship.

[0272] Figure 14 In example (1), the display device 1, the polarization separation component 101B, and the retroreflector 2 are made to... Figure 2C The example is configured more closely. This allows for a more compact overall optical system design. However, the space-suspended image 3A... Figure 14The amount by which the optical system of (1) floats is not large. For example, as an indicator of the amount by which the spatial levitation image 3A floats from the optical system, the figure shows the distance from the position where the light from the center of the image light is reflected on the polarization separation component 101B to the position where the spatial levitation image 3A is formed by the image light ( Figure 14 In example (1), it is L1).

[0273] In addition, regarding Figure 14 In the polarization design of the optical system of (1), the characteristics of P polarization and S polarization can also be interchanged. Specifically, the specified polarization of the image light emitted from the display device 1 can be made S polarization, and the characteristics of P polarization and S polarization can be interchanged regarding the reflection characteristics of the polarization separation component 101B. In this case, the P polarization and S polarization shown in the figure become opposite, but the optical design of the optical path, etc., can be implemented exactly the same.

[0274] Next, in Figure 14 Another example of the optical system and optical path of this embodiment is shown in (2). Figure 14 The optical system of (2) is designed to achieve [the desired result]. Figure 14 The optical system of (1) has the same compactness, while increasing the amount of space-suspended images floating from the optical system, and thus changing Figure 14 The structure of the optical system in (1). For Figure 14 (2) is marked with Figure 14 (1) The same reference numerals are used in the same figures, and repeated detailed descriptions are omitted.

[0275] Figure 14 In (2), with Figure 14 Similarly, (1) the image light of a predetermined polarization (P-polarization in the figure) emitted from the display device 1 travels vertically from the image display surface of the display device 1. Here, the polarization characteristics of the polarization separation component 101B are configured in accordance with... Figure 14 The phase difference between (1) and (2) is 90 degrees. Image light of a specified polarization (P polarization in the figure) traveling vertically from the image display surface of the display device 1 passes through the polarization separation member 101B.

[0276] Here, with Figure 14 Unlike (1), in front of the image light passing through the polarization separation component 101B, instead of a retroreflector 2 with a λ / 4 wave plate 21 attached, a specular reflector 4 with a λ / 4 wave plate 21B attached is arranged. Here, the reflection on the specular reflector 4 is specular reflection (also called orthographic reflection), not retroreflection.

[0277] Therefore, the image light passing through the polarization separation component 101B undergoes specular reflection on the mirror reflector 4 to which the λ / 4 waveplate 21B is attached. The image light, after being specularly reflected on the mirror reflector 4 and traveling again towards the polarization separation component 101B, passes through the λ / 4 waveplate 21 twice, thus changing from the predetermined polarized light (P-polarized light in the figure) emitted from the display device 1 to another polarized light (S-polarized light in the figure). The image light traveling again towards the polarization separation component 101B is now another polarized light (S-polarized light in the figure), and therefore reflects off the polarization separation component 101B.

[0278] Here, Figure 14 The orientation of the polarization separation component 101B in (2) is related to... Figure 14 Because (1) is different, the image light reflected from the polarization separation component 101B travels in the opposite direction to the position where the user should be located. A retroreflector 2 with a λ / 4 waveplate 21C attached is positioned in front of the image light reflected from the polarization separation component 101B. The image light is retroreflected by the retroreflector 2. The image light, retroreflected by the retroreflector 2 and traveling again towards the polarization separation component 101B, passes through the λ / 4 waveplate 21C twice, thereby changing from another polarized light (S-polarized light in the figure) back to the specified polarized light (P-polarized light in the figure).

[0279] The image light that travels again towards the polarization separation component 101B is predefined polarized light (P-polarized light in the figure), so it travels through the polarization separation component 101B to the position where the user should be. The image light passing through the polarization separation component 101B forms a spatial levitation image 3B. The spatial levitation image 3B can be viewed well by the user from the direction of arrow A.

[0280] Here, Figure 14 In (2), it is also related to Figure 14 Similarly, regarding the retroreflective characteristics of the retroreflector 2, there exists a relationship in which the optical path length of the image light emitted from the display device 1 to the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 to the formation position of the spatially suspended image 3B. The formation position of the spatially suspended image 3B in the direction of travel of the image light passing through the polarization separation component 101B is determined by this relationship.

[0281] Figure 14 The optical path length of the image light emitted from the display device 1 to the retroreflector 2 in (2) is related to... Figure 14 The optical path length of the image light emitted from the display device 1 to the retroreflector 2 in (1) is longer. This is because... Figure 14 In the optical system of (2), the optical path length for the image light emitted from the display device 1 to reach the retroreflector 2 is increased by Figure 14The optical path that does not exist in the optical system of (1) is the reciprocating optical path between the polarization separation component 101B and the mirror reflector 4.

[0282] thus, Figure 14 In the optical system of (2), the distance from the position of the light ray passing through the polarization separation component 101B from the center part of the image light to the position of the spatially suspended image 3B formed by the image light is ( Figure 14 In example (2), it is L2), and Figure 14 In the optical system of (1), the distance from the position where the light ray from the center of the image light is reflected on the polarization separation component 101B to the position where the spatially suspended image 3A is formed by the image light is ( Figure 14 In example (1), L1 is much longer.

[0283] In addition, regarding Figure 14 In the polarization design of the optical system of (2), the characteristics of P polarization and S polarization can also be interchanged. Specifically, the specified polarization of the image light emitted from the display device 1 can be made S polarization, and the characteristics of P polarization and S polarization can be interchanged regarding the reflection characteristics of the polarization separation component 101B. In this case, the P polarization and S polarization shown in the figure become opposite, but the optical design of the optical path, etc., can be achieved exactly the same.

[0284] According to the above description of Embodiment 2 of the present invention Figure 14 (1) and Figure 14 The optical system of (2) enables a more compact optical system. In particular, according to Figure 14 The optical system of (2) is able to make the amount of space-suspended images float up from the optical system by a greater amount with a more compact optical system.

[0285] In addition, in order to Figure 14 (1) or Figure 14 When the optical system of (2) is installed in the space levitation image display device, simply replace the optical system in the space levitation image display device described in Example 1 with Figure 14 (1) or Figure 14 The optical system of (2) can achieve this. Specifically, it can be... Figure 14 The optical system of (1) and Figure 4E , Figure 4F , Figure 4G , Figure 4H , Figure 4I , Figure 4J , Figure 4K , Figure 4L or Figure 4M The optical system of the spatial levitation image display device is replaced. In this case, because the optical system becomes more compact, the housing of the spatial levitation image display device in each figure can be made smaller.

[0286] In addition, specifically, it can be Figure 14 The optical system of (2) and Figure 4E , Figure 4F , Figure 4G , Figure 4K or Figure 4L The optical system of the levitation image display device is replaced. In this case, the amount by which the levitation image rises from the optical system is greater. In addition, since the optical system becomes more compact, the housing of the levitation image display device in each figure can be made smaller.

[0287] <Example 3>

[0288] As Embodiment 3 of the present invention, another structural example of a spatial levitation image display device will be described, which is capable of displaying multi-layered spatial levitation images with varying amounts of levitation from the optical system. Furthermore, the differences from Embodiment 1 or Embodiment 2 will be described in this embodiment, and repeated descriptions of structures identical to those in Embodiment 1 or Embodiment 2 will be omitted. In the following description of this embodiment, it will be specified that polarization (polarized light) and another polarization (polarized light) are polarizations (polarized light) with a phase difference of 90°.

[0289] Figure 15A This is an example of the structure and optical path of the optical system of a spatial levitation image display device that displays multi-layered spatial levitation images. Figure 15A The optical system in the display device that serves as the image source has only one display device 1. Figure 15A In the example, two display areas, display area 1501 and display area 1502, are set on the display screen of display device 1. Figure 15A The optical system displays a 3D spatial levitation image corresponding to the display area 1501. Figure 15A The optical system displays the spatially suspended image 3E corresponding to the display area 1502.

[0290] Figure 15A In the example, when the user views the 3D and 3E spatial levitation images from the direction of arrow A, the 3D spatial levitation image appears to be displayed in front of the 3E spatial levitation image (closer to the user). From the user's perspective, the 3D and 3E spatial levitation images appear to overlap, so these spatial levitation images appear to have a depth of 2 layers.

[0291] Next, the spatial levitation image with two layers of depth was realized. Figure 15AThe detailed structure of the optical system will be described. The structure of display device 1 is the same as that of Embodiment 1, so repeated descriptions are omitted. First, image light with a predetermined polarization (P-polarized in the figure) is output from display device 1. Image light with a predetermined polarization (P-polarized in the figure) is output at either display area 1501 or display area 1502, but... Figure 15A In the optical system, a λ / 2 waveplate 22 is attached in such a way that it includes the display area 1502. Therefore, the image light emitted from the display area 1502 is converted into another polarized light (S polarized light in the figure) after passing through the λ / 2 waveplate 22.

[0292] Here, the image light with a specified polarization (P-polarized in the figure) output from the display area 1501 travels as shown and is incident on the polarization separation component 101D. The polarization separation component 101D selectively transmits the specified polarized light (P-polarized light in the figure) and reflects the other polarized light (S-polarized light in the figure).

[0293] Therefore, the image light with a predetermined polarization (P-polarized in the figure) output from the display area 1501 passes through the polarization separation member 101D and reaches the retroreflector 2D to which the λ / 4 waveplate 21D is attached. The image light, which is retroreflected on the retroreflector 2D and travels back to the polarization separation member 101D, passes through the λ / 4 waveplate 21D twice, thereby changing from the predetermined polarized light (P-polarized light in the figure) emitted from the display device 1 to another polarized light (S-polarized light in the figure).

[0294] The image light traveling towards the polarization separator 101D again is of a different polarization (S-polarized light in the figure), so it is reflected on the polarization separator 101D toward the position where the user should be located. The direction of travel of the image reflected on the polarization separator 101D is determined based on the configuration angle of the polarization separator 101D. Figure 15A In the example, the image light traveling toward the polarization separation component 101D is reflected at a right angle onto the polarization separation component 101D and travels as shown in the figure. The image light reflected onto the polarization separation component 101D forms a spatially suspended 3D image.

[0295] Next, the image light output from display area 1502 is converted into another polarized light (S-polarized light in the figure) by the λ / 2 waveplate 22 and travels before being incident on polarization separation member 101E. Polarization separation member 101E selectively transmits the specified polarized light (P-polarized light in the figure) and reflects the other polarized light (S-polarized light in the figure). Therefore, the image light of the other polarization (S-polarized light in the figure) output from display area 1502 and transmitted through the λ / 2 waveplate 22 is reflected on polarization separation member 101E and reaches the retroreflector 2E to which the λ / 4 waveplate 21E is attached. The direction of travel of the image reflected on polarization separation member 101E is determined based on the configuration angle of polarization separation member 101E. Figure 15A In the example, the image light traveling toward the polarization separation member 101E is reflected at a right angle onto the polarization separation member 101E and travels as shown in the figure. The image light, which is retroreflected on the retroreflector 2E and travels toward the polarization separation member 101E again, passes through the λ / 4 waveplate 21E twice, thereby changing from another polarized light (S-polarized light in the figure) to a specified polarized light (P-polarized light in the figure).

[0296] The image light traveling again towards the polarization separation member 101E is predefined polarized light (P-polarized light in the figure), so it passes through the polarization separation member 101E. As shown, the image light passing through the polarization separation member 101E travels towards the polarization separation member 101D. As described above, the polarization separation member 101D selectively transmits the predefined polarized light (P-polarized light in the figure) and reflects another type of polarized light (S-polarized light in the figure). Therefore, the image light from the retroreflector 2E, which is the predefined polarized light (P-polarized light in the figure), travels through the polarization separation member 101D towards the position where the user should be located. The image light passing through the polarization separation member 101D forms a spatially suspended image 3E.

[0297] Figure 15A In the example, a light shield is provided between the optical paths of the image light output from the display area 1501 and the image light output from the display area 1502 in order to prevent the image light of each from leaking into the optical path of the other.

[0298] Figure 15A In this example, both polarization separation components 101D and 101E are arranged at a 45-degree angle relative to the direction of travel of the image light from the display device 1. As a result, the image light forming the spatial levitation image 3D and the image light forming the spatial levitation image 3E travel in the same direction (aligned) towards the position where the user should be located. Because of this structure, in Figure 15AIn the example, when a user views the spatial levitation image 3D and spatial levitation image 3E from the direction of arrow A (y-direction), the spatial levitation image 3D, spatial levitation image 3E, polarization separation component 101D, polarization separation component 101E, and retroreflector 2E are arranged in a straight line from the user's perspective (e.g., Figure 15A In the example, it is a straight line of light from the retroreflector 2E to the spatially suspended image 3E, that is, a straight line extending in the direction of the user.

[0299] Furthermore, at this time, the display device 1 and the retroreflector 2D are positioned at a location offset from the same straight line. Additionally, Figure 15A In the example, the positions of polarization separation components 101D and 101E are determined such that when a user views the spatial levitation image 3D and spatial levitation image 3E from the direction of arrow A (y-direction), the center of the image in the left-right direction (x-direction) of spatial levitation image 3D coincides with the center of the image in the left-right direction (x-direction) of spatial levitation image 3E. When the center of the image in the left-right direction (x-direction) of spatial levitation image 3D coincides with the center of the image in the left-right direction (x-direction) of spatial levitation image 3E from the user's perspective, it is easier for the user to view, and the image content creator does not need to consider offset, so it is better. In addition, the optical layout is also simpler, which is better.

[0300] In addition, regarding Figure 15A In the polarization design of the optical system, the characteristics of P-polarization and S-polarization can also be interchanged. Specifically, the image light emitted from the display area 1501 of the display device 1 can be designated as S-polarization, and the other polarization emitted from the display area 1502 of the display device 1 and transmitted through the λ / 2 waveplate 22 can be designated as P-polarization. The characteristics of P-polarization and S-polarization are interchanged with respect to the reflection characteristics of the polarization separation member 101D and the polarization separation member 101E. In this case, the P-polarization and S-polarization shown in the figure become opposite, but the optical design of the optical path, etc., can be implemented exactly the same.

[0301] Based on the above explanation Figure 15A The optical system can form a spatially suspended image with two layers of depth using a single display device. Alternatively, a structure can be adopted that includes display devices corresponding to display areas 1501 and 1502 respectively. However, when using a structure with multiple display devices, the corresponding circuitry increases, potentially leading to higher costs. Therefore, by means of... Figure 15A The diagram shows that a spatial levitation image with two layers of depth can be formed using only one display device, enabling the formation of an optical system with two layers of depth at a lower cost.

[0302] In addition, in order to Figure 15A When the optical system is installed in the space-suspended image display device, simply replace the optical system in the space-suspended image display device described in Example 1 with... Figure 15A This can be achieved using optical systems. Specifically, it can be... Figure 15A optical system and Figure 4E , Figure 4F , Figure 4G , Figure 4K or Figure 4L The optical system of the spatial levitation image display device is replaced. In this case, a spatial levitation image display device with a two-layer depth of spatial levitation images can be formed in each figure. In particular, in Figure 4K and Figure 4L In this case, a spatially suspended image with two layers of depth can be formed on the front side of the transmissive self-emissive image display device 1650 as viewed from the user. In this case, the image with two layers of depth and the transmissive self-emissive image display device 1650 can be combined to form a three-layer image with different depths that the user can see.

[0303] Next, for another example of the optical system and optical path of the spatial levitation image display device in Embodiment 3, which displays multi-layered spatial levitation images with different amounts of levitation from the optical system, using... Figure 15B Please provide an explanation. Figure 15B The optical system is Figure 15A This is a variant example where a portion of the optical system's structure has been modified. Therefore, Figure 15B The examples illustrate the relationship with Figure 15A The differences, for the same Figure 15A The same structure is used, omitting repeated explanations.

[0304] Figure 15B In optical systems, with Figure 15A Similarly, two display areas, display area 1501 and display area 1502, are provided on the display screen (i.e., the display screen) of display device 1. However, no λ / 2 waveplate is attached to the exit surface of display area 1502. Figure 15B The characteristics and configuration of polarization separation components 101D and 101E in the optical system are similar to Figure 15A The optical system is the same. Figure 15A The optical system is equipped with a retroreflector 2D with a λ / 4 wave plate 21D bonded thereon and a retroreflector 2E with a λ / 4 wave plate 21E bonded thereon. In contrast, Figure 15B The optical system is equipped with only one retroreflector 2 with a λ / 4 wave plate 21 attached. Figure 15B In the optical system, a λ / 2 waveplate 22 is arranged in the optical path from the polarization separation component 101E to the polarization separation component 101D.

[0305] Here, Figure 15B In the optical system, the optical path and various optical elements of the image light emitted from the display area 1501 with a specified polarization (P polarization in the figure) up to the formation of the spatially suspended image 3D are identical except that the retroreflector 2D with the λ / 4 wave plate 21D attached is replaced with the retroreflector 2 with the λ / 4 wave plate 21 attached, so the description is omitted.

[0306] Here, Figure 15B In the optical system, a predetermined polarized light (P-polarized light in the figure) emitted from the display area 1502 travels towards the polarization separation member 101E and is incident on the polarization separation member 101E. The polarization separation member 101E selectively transmits the predetermined polarized light (P-polarized light in the figure) and reflects another polarized light (S-polarized light in the figure). Therefore, the image light traveling towards the polarization separation member 101E passes through the polarization separation member 101E and travels towards the retroreflector 2. The image light, retroreflected on the retroreflector 2 and traveling back towards the polarization separation member 101E, passes through the λ / 4 waveplate 21 twice, thereby changing from the predetermined polarized light (P-polarized light in the figure) to another polarized light (S-polarized light in the figure).

[0307] The image light traveling again towards the polarization separation member 101E is of a different polarization (S-polarized light in the figure), so it is reflected by the polarization separation member 101E and travels towards the λ / 2 waveplate 22. The image light incident on the λ / 2 waveplate 22 passes through the λ / 2 waveplate 22, thereby changing from the other polarization (S-polarized light in the figure) to the specified polarization (P-polarized light in the figure). The image light passing through the λ / 2 waveplate 22 travels towards the polarization separation member 101D. The polarization separation member 101D selectively transmits the specified polarization (P-polarized light in the figure) and reflects the other polarization (S-polarized light in the figure). Therefore, the image light from the λ / 2 waveplate 22, as the specified polarization (P-polarized light in the figure), travels through the polarization separation member 101D towards the position where the user should be located. The image light passing through the polarization separation member 101D forms a spatially suspended image 3E.

[0308] Figure 15B In the example, a light shield is provided between the optical paths of the image light output from the display area 1501 and the image light output from the display area 1502 in order to prevent the image light of each from leaking into the optical path of the other.

[0309] Figure 15BIn this example, both polarization separation components 101D and 101E are arranged at a 45-degree angle relative to the direction of travel of the image light from the display device 1. As a result, the image light forming the spatial levitation image 3D and the image light forming the spatial levitation image 3E travel in the same direction (aligned) toward the position where the user should be located.

[0310] Because of this structure, therefore Figure 15B In the example, when a user views the spatial levitation image 3D and spatial levitation image 3E from the direction of arrow A (y-direction), the spatial levitation image 3D, spatial levitation image 3E, polarization separation component 101D, λ / 2 waveplate 22, and polarization separation component 101E are arranged on the same straight line from the user's perspective (e.g., Figure 15B In the example, it is a straight line of light from the polarization separation component 101E to the spatial levitation image 3E (i.e., a straight line extending in the direction of the user). In addition, at this time, the display device 1 and the retroreflector 2 are positioned at a position offset from the position on the same straight line.

[0311] in addition, Figure 15B In the example, the positions of polarization separation components 101D and 101E are determined such that when a user views the spatial levitation image 3D and spatial levitation image 3E from the direction of arrow A (y-direction), the center of the image in the left-right direction (x-direction) of spatial levitation image 3D coincides with the center of the image in the left-right direction (x-direction) of spatial levitation image 3E. When the center of the image in the left-right direction (x-direction) of spatial levitation image 3D coincides with the center of the image in the left-right direction (x-direction) of spatial levitation image 3E from the user's perspective, it is easier for the user to view, and the image content creator does not need to consider offset, so it is better. In addition, the optical layout is also simpler, which is better.

[0312] In addition, regarding Figure 15B In the polarization design of the optical system, the characteristics of P-polarization and S-polarization can also be interchanged. Specifically, the predetermined polarization of the image light emitted from the display area 1501 of the display device 1 can be S-polarized, and the predetermined polarization of the image light emitted from the display area 1502 can also be S-polarized. The characteristics of P-polarization and S-polarization are interchanged regarding the reflection characteristics of the polarization separation member 101D and the polarization separation member 101E. In this case, the P-polarization and S-polarization shown in the figure become opposite, but the optical design of the optical path, etc., can be implemented exactly the same.

[0313] The above explanation Figure 15B In the optical system, the optical path length from the image light emitted from the display area 1501 to the point where the spatially suspended 3D image is formed is... Figure 15A The optical system is the same. Additionally... Figure 15BIn the optical system, the optical path length from the image light emitted from the display area 1502 to the spatially suspended image 3E is... Figure 15A The optical system is the same. Therefore, the formation positions of the 3D spatial levitation image and the 3E spatial levitation image are also the same. Figure 15A The optical system is the same.

[0314] in addition, Figure 15B In the example shown, the optical path length from the image light emitted from the display area 1501 to the spatially suspended image 3D is the same as the optical path length from the image light emitted from the display area 1502 to the spatially suspended image 3E. Figure 15A The retroreflector 2D and retroreflector 2E are respectively configured in the optical system. Figure 15B The optical system consists of a single retroreflector 2. Retroreflectors are high-cost components, so using only one retroreflector reduces costs. Therefore, according to... Figure 15B The optical system is more capable than Figure 15A The optical system enables the formation of spatially suspended images with two layers of depth at a lower cost.

[0315] Next, for another example of the optical system and optical path of the spatial levitation image display device in Embodiment 3, which displays multi-layered spatial levitation images with different amounts of levitation from the optical system, using... Figure 16A Please provide an explanation. Figure 16A The optical system is Figure 15A This is a variant example where a portion of the optical system's structure has been modified. Therefore, Figure 16A The examples illustrate the relationship with Figure 15A The differences, for the same Figure 15A The same structure is used, omitting repeated explanations.

[0316] Figure 16A In the optical system, a position centered between display area 1501 and display area 1502 in the display screen of display device 1 is used to align display device 1 with... Figure 15A The optical system configuration is compared to a tilted configuration. The illustration shows an example where the tilt of display device 1 is 30 degrees. In the illustration, the length of the display screen of display device 1 is extended corresponding to the tilt. The tilt of display device 1 results in a longer optical path length for the image light emitted from display area 1501 to reach retroreflector 2D compared to... Figure 15A The configuration of the optical system is shorter, which determines the direction. Therefore, 3D spatial levitation images and... Figure 15A The optical system is formed on the deeper side as seen from the user. Furthermore, the tilt of the display device 1 results in a longer optical path for the image light emitted from the display area 1502 to reach the retroreflector 2E compared to... Figure 15AThe optical system configuration is longer. Space-suspended image 3E and Figure 15A The optical system is formed on the front side as seen from the user.

[0317] therefore, Figure 16A In the optical system, the depth distance between the two layers of spatial levitation images, 3D and 3E, can be increased. Figure 15A The optical system is closer. That is, Figure 16A In the example, due to the tilt of the display device 1, the optical path length of the image light emitted from the display area 1501 to the retroreflector 2D is shorter than the optical path length of the image light emitted from the display area 1502 to the retroreflector 2E. Furthermore, due to the tilt of the display device 1 relative to... Figure 15A The optical system is configured at an angle, and the resulting two layers of spatial levitation images, 3D and 3E, are also relative to each other. Figure 15A The optical system is configured at an angle. In the display device 1 relative to... Figure 15A With the optical system tilted at 30 degrees, the two spatial levitation images, spatial levitation image 3D and spatial levitation image 3E, are also tilted at 30 degrees.

[0318] Here, Figure 16A In the optical system, an angle control sheet 23 can also be attached to the surface of the absorptive polarizer 12, corresponding to the tilted arrangement of the display device 1. The angle control sheet 23 is a sheet that controls the direction of light travel and deflects it by a predetermined angle. Specifically, this sheet can be implemented using a linear Fresnel lens. If the light intensity is strongest in the normal direction of the surface of the liquid crystal display panel 11 in the angular distribution of the image light from the liquid crystal display panel 11, then by controlling the angle of light travel in a way that counteracts the tilt of the display device 1, the light utilization efficiency of the optical system can be improved.

[0319] When the tilt of the display device 1 is 30 degrees, an angle control plate 23 capable of changing the light travel angle by 30 degrees can be used to counteract this tilt. When using the angle control plate 23, a λ / 2 waveplate 22 can be attached to the surface of the angle control plate 23 for the display area 1502. Furthermore, the angle control plate 23 can be used when it is necessary to improve the light utilization efficiency of the optical system; it can also be used without the angle control plate 23. Figure 16A The optical system.

[0320] in addition, Figure 16A In the optical system, the details of the optical path of the image light emitted from display area 1501 and display area 1502 to form the spatial levitation image 3D and spatial levitation image 3E, and the details of each optical element are as follows: Figure 15A Since the optical systems are the same, repeated descriptions are omitted.

[0321] In addition, regarding Figure 16A In the polarization design of the optical system, the characteristics of P-polarization and S-polarization can also be interchanged. Specifically, the image light emitted from the display area 1501 of the display device 1 can be designated as S-polarization, and the other polarization emitted from the display area 1502 of the display device 1 and transmitted through the λ / 2 waveplate 22 can be designated as P-polarization. The characteristics of P-polarization and S-polarization are interchanged with respect to the reflection characteristics of the polarization separation member 101D and the polarization separation member 101E. In this case, the P-polarization and S-polarization shown in the figure become opposite, but the optical design of the optical path, etc., can be implemented exactly the same.

[0322] in addition, Figure 16A Similarly, in the example, when a user views the spatial levitation image 3D and spatial levitation image 3E from the direction of arrow A (y direction), the spatial levitation image 3D, spatial levitation image 3E, polarization separation component 101D, polarization separation component 101E, and retroreflector 2E are arranged in a straight line from the user's perspective (e.g., Figure 16A In the example, it is a straight line of light from the retroreflector 2E to the spatial levitation image 3E (i.e., a straight line extending towards the user). Furthermore, at this time, the display device 1 and the retroreflector 2D are positioned at a location offset from this straight line.

[0323] By using the above instructions Figure 16A The optical system enables a spatial levitation image display device that brings the depth distance between two layers of spatial levitation images closer together.

[0324] Next, for another example of the optical system and optical path of the spatial levitation image display device in Embodiment 3, which displays multi-layered spatial levitation images with different amounts of levitation from the optical system, using... Figure 16B Please provide an explanation. Figure 16B The optical system is Figure 15B This is a variant example where a portion of the optical system's structure has been modified. Therefore, Figure 16B The examples illustrate the relationship with Figure 15B The differences, for the same Figure 15B The same structure is used, omitting repeated explanations.

[0325] Figure 16B In the optical system, a position centered between display area 1501 and display area 1502 in the display screen of display device 1 is used to align display device 1 with... Figure 15BThe optical system configuration is compared to a tilted configuration. The illustration shows an example where the tilt of display device 1 is 30 degrees. In the illustration, the length of the display screen of display device 1 is extended corresponding to the tilt. The tilt of display device 1 results in a longer optical path length for the image light emitted from display area 1501 to reach retroreflector 2 compared to... Figure 15B The configuration of the optical system is shorter, which determines the direction. Therefore, 3D spatial levitation images and... Figure 15B The optical system is formed on the deeper side as seen from the user. Furthermore, the tilt of the display device 1 ensures that the optical path length of the image light emitted from the display area 1502 to the retroreflector 2 is longer than... Figure 15B The configuration of the optical system is determined by its longer shape. Space-suspended image 3E and Figure 15A The optical system is formed on the front side as seen from the user.

[0326] therefore, Figure 16B In the optical system, the depth distance between the two layers of spatial levitation images, 3D and 3E, can be increased. Figure 15B The optical system is closer. That is, Figure 16B In the example, due to the tilt of the display device 1, the optical path length of the image light emitted from the display area 1501 to the retroreflector 2 is shorter than the optical path length of the image light emitted from the display area 1502 to the retroreflector 2.

[0327] Furthermore, since the display device 1 is relative to Figure 15B The optical system is configured at an angle, and the resulting two layers of spatial levitation images, 3D and 3E, are also relative to each other. Figure 15B The optical system is configured at an angle. In the display device 1 relative to... Figure 15B With the optical system tilted at 30 degrees, the two spatial levitation images, spatial levitation image 3D and spatial levitation image 3E, are also tilted at 30 degrees.

[0328] Here, Figure 16B In optical systems, it can also be used with Figure 16A Similarly, in the optical system, corresponding to the tilted display device 1, an angle control sheet 23 is attached to the surface of the absorptive polarizer 12. If the light intensity is strongest in the normal direction of the surface of the liquid crystal display panel 11 in the angular distribution of the image light from the liquid crystal display panel 11, then by controlling the light travel angle in a way that counteracts the tilt of the display device 1, the light utilization efficiency of the optical system can be improved. When the tilt of the display device 1 is 30 degrees, an angle control sheet 23 that can change the light travel angle by 30 degrees can be used to counteract this tilt. Furthermore, the angle control sheet 23 can be used whenever it is necessary to improve the light utilization efficiency of the optical system, and it can also be configured without using the angle control sheet 23. Figure 16B The optical system.

[0329] in addition, Figure 16B In the optical system, the details of the optical path of the image light emitted from display area 1501 and display area 1502 to form the spatial levitation image 3D and spatial levitation image 3E, and the details of each optical element are as follows: Figure 15A Since the optical systems are the same, repeated descriptions are omitted.

[0330] In addition, regarding Figure 16B In the polarization design of the optical system, the characteristics of P-polarization and S-polarization can also be interchanged. Specifically, the predetermined polarization of the image light emitted from the display area 1501 of the display device 1 can be S-polarized, and the predetermined polarization of the image light emitted from the display area 1502 can also be S-polarized. The characteristics of P-polarization and S-polarization are interchanged regarding the reflection characteristics of the polarization separation member 101D and the polarization separation member 101E. In this case, the P-polarization and S-polarization shown in the figure become opposite, but the optical design of the optical path, etc., can be implemented exactly the same.

[0331] By using the above instructions Figure 16B The optical system enables a spatial levitation image display device that brings the depth distance between two layers of spatial levitation images closer together.

[0332] Next, for another example of the optical system and optical path of the spatial levitation image display device in Embodiment 3, which displays multi-layered spatial levitation images with different amounts of levitation from the optical system, using... Figure 17A Please provide an explanation.

[0333] Figure 17A This is an example of the structure and optical path of the optical system of a spatial levitation image display device that displays multi-layered spatial levitation images. Figure 17A The optical system in the display device that serves as the image source has only one display device 1. Figure 17A In the example, two display areas, display area 1501 and display area 1502, are set on the display screen of display device 1. Figure 17A The optical system displays a 3D spatial levitation image corresponding to the display area 1501. Figure 17A The optical system displays the spatially suspended image 3E corresponding to the display area 1502.

[0334] Figure 17A In the example, when a user views the levitation images 3E and 3D from the direction of arrow A, levitation image 3E appears to be displayed in front of levitation image 3D. From the user's perspective, levitation images 3E and 3D appear to overlap, so these levitation images appear to have a two-layer depth.

[0335] Next, regarding Figure 17A The detailed structure of the optical system will be described. The structure of the display device 1 is the same as that of Embodiment 1, so repeated descriptions are omitted. First, image light with a specified polarization (P-polarized in the figure) is output from the display device 1. Image light with a specified polarization (P-polarized in the figure) is output at either the display area 1501 or the display area 1502.

[0336] Here, the image light with a specified polarization (P-polarized in the figure) output from the display area 1501 travels as shown and is incident on the polarization separation member 101D. The polarization separation member 101D selectively transmits the specified polarized light (P-polarized in the figure) and reflects the other polarized light (S-polarized in the figure). Therefore, the image light with the specified polarization (P-polarized in the figure) output from the display area 1501 passes through the polarization separation member 101D and reaches the retroreflector 2D to which the λ / 4 waveplate 21D is attached. The image light, retroreflected on the retroreflector 2D and traveling back to the polarization separation member 101D, passes through the λ / 4 waveplate 21D twice, thereby changing from the specified polarized light (P-polarized in the figure) emitted from the display device 1 to the other polarized light (S-polarized in the figure).

[0337] The image light traveling towards the polarization separator 101D again is of a different polarization (S-polarized light in the figure), so it is reflected on the polarization separator 101D toward the position where the user should be located. The direction of travel of the image reflected on the polarization separator 101D is determined based on the configuration angle of the polarization separator 101D. Figure 17A In the example, the image light traveling toward the polarization separation component 101D is reflected at a right angle onto the polarization separation component 101D and travels as shown in the figure. The image light reflected onto the polarization separation component 101D forms a spatially suspended 3D image.

[0338] Next, the image light with a predetermined polarization (P-polarized in the figure) emitted from the display area 1502 travels as shown and is incident on the polarization separation member 101E. The polarization separation member 101E selectively transmits the predetermined polarized light (P-polarized in the figure) and reflects the other polarized light (S-polarized in the figure). Therefore, the image light traveling toward the polarization separation member 101E passes through the polarization separation member 101E and travels toward the mirror reflector 24 to which the λ / 4 waveplate 21F is attached. The image light, after being mirror-reflected on the mirror reflector 24 and traveling toward the polarization separation member 101E again, passes through the λ / 4 waveplate 21F twice, thereby changing from predetermined polarized light (P-polarized in the figure) to the other polarized light (S-polarized in the figure).

[0339] The image light traveling again towards the polarization separation component 101E is of a different polarization (S-polarized light in the figure), so it is reflected by the polarization separation component 101E and reaches the retroreflector 2E to which the λ / 4 waveplate 21E is attached. The direction of travel of the image reflected on the polarization separation component 101E is determined by the configuration angle of the polarization separation component 101E. Figure 17A In the example, the image light traveling toward the polarization separation member 101E is reflected at a right angle onto the polarization separation member 101E and travels as shown in the figure. The image light, which is retroreflected on the retroreflector 2E and travels toward the polarization separation member 101E again, passes through the λ / 4 waveplate 21E twice, thereby changing from another polarized light (S-polarized light in the figure) to a specified polarized light (P-polarized light in the figure).

[0340] The image light traveling again towards the polarization separation member 101E is predefined polarized light (P-polarized light in the figure), so it passes through the polarization separation member 101E. As shown, the image light passing through the polarization separation member 101E travels towards the polarization separation member 101D. As described above, the polarization separation member 101D selectively transmits the predefined polarized light (P-polarized light in the figure) and reflects another type of polarized light (S-polarized light in the figure). Therefore, the image light from the retroreflector 2E, which is the predefined polarized light (P-polarized light in the figure), travels through the polarization separation member 101D towards the position where the user should be located. The image light passing through the polarization separation member 101D forms a spatially suspended image 3E.

[0341] Figure 17A In the example, a light shield is provided between the optical paths of the image light output from the display area 1501 and the image light output from the display area 1502 in order to prevent the image light of each from leaking into the optical path of the other.

[0342] Figure 17A In the example, polarization separation component 101D is configured to tilt 45 degrees relative to the direction of travel of the image light from display device 1. Polarization separation component 101E is configured to tilt 45 degrees relative to the direction of travel of the image light from display device 1 in a different direction than polarization separation component 101D.

[0343] Therefore, the image light forming the spatial levitation image 3E and the image light forming the spatial levitation image 3D travel in the same direction (aligned) towards the position where the user should be. Because of this structure, in Figure 17A In the example, when a user views the spatial levitation image 3E and spatial levitation image 3D from the direction of arrow A (y-direction), the spatial levitation image 3E, spatial levitation image 3D, polarization separation component 101D, polarization separation component 101E, and retroreflector 2E are arranged on the same straight line from the user's perspective (e.g., Figure 17AIn the example, it is a straight line of light from the retroreflector 2E to the spatially suspended image 3E, that is, a straight line extending in the direction of the user.

[0344] Furthermore, at this time, the display device 1, the retroreflector 2D, and the specular reflector 24 are positioned offset from the position on the same straight line. Additionally, Figure 17A In the example, the positions of polarization separation components 101D and 101E are determined such that when a user views spatial levitation image 3E and spatial levitation image 3D from the direction of arrow A (y direction), the center of the image in the left-right direction (x direction) of spatial levitation image 3E is consistent with the center of the image in the left-right direction (x direction) of spatial levitation image 3D.

[0345] When the center of the image in the left-right direction (x-direction) of the spatial levitation image 3E is aligned with the center of the image in the left-right direction (x-direction) of the spatial levitation image 3D from the user's perspective, it is easier for the user to view, and the producers of the image content do not need to consider offset, so it is better. In addition, the optical layout is also simpler, which is better.

[0346] In addition, regarding Figure 17A In the polarization design of the optical system, the characteristics of P-polarization and S-polarization can also be interchanged. Specifically, the predetermined polarization of the image light emitted from the display area 1501 of the display device 1 can be S-polarized, and the predetermined polarization of the light emitted from the display area 1502 of the display device 1 can also be S-polarized. The characteristics of P-polarization and S-polarization are interchanged regarding the reflection characteristics of the polarization separation member 101D and the polarization separation member 101E. In this case, the P-polarization and S-polarization shown in the figure become opposite, but the optical design of the optical path, etc., can be implemented exactly the same.

[0347] Based on the above explanation Figure 17A The optical system can form a spatially suspended image with two layers of depth using a single display device. Alternatively, a structure can be adopted that includes display devices corresponding to display areas 1501 and 1502 respectively. However, when using a structure with multiple display devices, the corresponding circuitry increases, potentially leading to higher costs. Therefore, by means of... Figure 17A The diagram shows that a spatial levitation image with two layers of depth can be formed using only one display device, enabling the formation of an optical system with two layers of depth at a lower cost.

[0348] Additionally, here, for Figure 17AThe optical path lengths in the optical system from the image light emitted from the display area 1501 of the display device 1 to the spatially suspended image 3D, and from the image light emitted from the display area 1502 of the display device 1 to the spatially suspended image 3E, will be explained. These optical path lengths will be explained using the optical path lengths of light rays emitted from the center of the display area 1501 along the normal direction, and the optical path lengths of light rays emitted from the center of the display area 1502 along the normal direction, respectively. The following explanation follows the same principle.

[0349] first, Figure 17A In the optical system, the optical path length of the image light emitted from the display area 1501 of the display device 1 to the polarization separation component 101D is equal to the optical path length of the image light emitted from the display area 1502 of the display device 1 to the polarization separation component 101E. This is consistent with... Figure 15A optical system and Figure 15B The same applies to optical systems.

[0350] Here, Figure 15A optical system and Figure 15B In the optical system, for the spatially suspended image 3E formed by the image light emitted from the display area 1502 of the display device 1, in order to sufficiently ensure the amount by which it floats from the optical system, it is necessary to extend the optical path length until the spatially suspended image 3E is formed by the image light emitted from the display area 1502 of the display device 1. As a result, it is necessary to ensure that the distance from the display device 1 to the polarization separation component 101D and the polarization separation component 101E is relatively long.

[0351] In contrast, Figure 17A In the optical system, the angle of the polarization separation component 101E is configured to differ from the angle of the polarization separation component 101D by 90 degrees. For the optical path from the image light emitted from the display area 1502 of the display device 1 until the spatially suspended image 3E is formed, an additional optical path is added between the polarization separation component 101E and the specular reflector 24. Therefore, Figure 17A The optical path length in the optical system from the image light emitted from the display area 1502 of the display device 1 to the spatially suspended image 3E is longer than... Figure 15A optical system and Figure 15B The optical path length of the image light emitted from the display area 1502 of the display device 1 to form the spatially suspended image 3E in the optical system is longer.

[0352] therefore, Figure 17A Even with Figure 15A optical system and Figure 15BCompared to the optical system, which shortens the distance from the display device 1 to the polarization separation components 101D and 101E, the spatially suspended image 3E formed by the image light emitted from the display area 1502 of the display device 1 can also be sufficiently ensured to float off the optical system by a sufficient amount.

[0353] Here, Figure 17A In the optical system, the optical path length from the image light emitted from the display area 1502 of the display device 1 to the formation of the spatially suspended image 3E can be changed depending on the distance D (d in the figure) between the mirror reflector 24 and the display surface of the display device 1. Therefore, in the spatially suspended image display device, the position of the mirror reflector 24 can be determined in a way that achieves the required levitation amount of the spatially suspended image 3E. In addition, the distance between the spatially suspended image 3E and the spatially suspended image 3D also changes depending on this distance D. Therefore, in the spatially suspended image display device that displays multiple spatially suspended images with different levitation amounts from the optical system, the position of the mirror reflector 24 can be determined in a way that achieves the required distance for the product by varying the distance between the multiple spatially suspended images.

[0354] Here, with Figure 15A optical system and Figure 15B Compared to optical systems, it can be seen that... Figure 17A In this optical system, the distance from the display device 1 to the polarization separation components 101D and 101E can be relatively short, thus reducing the overall size of the optical system. Furthermore, Figure 17A In the optical system, depending on the position of the mirror reflector 24, the distance between the spatial levitation image 3E and the spatial levitation image 3D can be set to a value greater than that of the mirror reflector 24. Figure 15A optical system and Figure 15B The optical system is shorter. That is, it uses... Figure 17A The optical system is capable of realizing a spatial levitation image display device that displays multi-layered spatial levitation images in a more compact form. Furthermore, it allows for setting the distance of the multi-layered spatial levitation images to the desired position using a simple structure, which is even more preferable.

[0355] In addition, in order to Figure 17A When the optical system is installed in the space-suspended image display device, simply replace the optical system in the space-suspended image display device described in Example 1 with... Figure 17A This can be achieved using optical systems. Specifically, it can be... Figure 17A optical system and Figure 4E , Figure 4F , Figure 4G , Figure 4K or Figure 4L The optical system of the space-suspended image display device was replaced.

[0356] In this case, a spatial levitation image display device with a two-layer depth can be realized in each figure. Specifically, in Figure 4K and Figure 4L In this case, a spatially suspended image with two layers of depth can be formed on the front side of the transmissive self-emissive image display device 1650 as viewed from the user. In this case, the image with two layers of depth and the transmissive self-emissive image display device 1650 can be combined to form a three-layer image with different depths that the user can see.

[0357] Next, for those who possess Figure 17A An example of a space-suspended image display device 1000 with an optical system, using Figure 17B Please provide an explanation. Figure 17B This is an example of the structure of a spatial levitation image display device 1000 that displays multi-layered spatial levitation images with different levitation levels. The spatial levitation image display device 1000 shown in the figure is equipped with... Figure 17A The optical system described in the text. Figure 17B An example is an example in which the optical system is configured such that the display device 1 and the mirror reflector 24 are opposite each other in the user's left and right directions (x direction). Figure 17B in, omit Figure 17A The accompanying drawings show the reference numerals for other elements in the optical system. For example... Figure 17B As shown, the spatial levitation image display device 1000 can display two layers of spatial levitation images, spatial levitation image 3E and spatial levitation image 3D, to the user 230. Furthermore, because... Figure 17A The optical system itself is relatively small, so the space levitation image display device 1000 can also be implemented in a relatively small size.

[0358] Based on the above explanation Figure 17B The 1000 spatial levitation image display device is a spatial levitation image display device that can display multi-layer spatial levitation images with different levitation amounts in a smaller size.

[0359] Next, for those who possess Figure 17A Another example of the space-suspended image display device 1000 with an optical system, using Figure 17C Please provide an explanation.

[0360] Figure 17C This is an example of the structure of a spatial levitation image display device 1000 that displays multi-layered spatial levitation images with different levitation levels. The spatial levitation image display device 1000 shown in the figure is equipped with... Figure 17A The optical system described in the text. Figure 17C An example is an example of an optical system configured such that the display device 1 and the mirror reflector 24 are opposite each other in the vertical direction (z direction) from the user's perspective. Figure 17C in, omit Figure 17A The accompanying drawings show the reference numerals for other elements in the optical system.

[0361] like Figure 17C As shown, the spatial levitation image display device 1000 can display two layers of spatial levitation images, spatial levitation image 3E and spatial levitation image 3D, to the user 230. Here, Figure 17C In this example, an airborne operation detection sensor can be set up to detect user actions performed on both the 3E and 3D levitation images. Specifically, such as... Figure 17C As shown, an airborne operation detection sensor 1351D is provided for detecting user actions on the 3D suspended image. Additionally, an airborne operation detection sensor 1351E is provided for detecting user actions on the 3E suspended image. For example, it can be... Figure 3 The air operation detection sensor 1351 in the middle is replaced with these two sensors. Figure 3 The air operation detection unit 1350 can determine, based on signals from these sensors, whether there are user operations performed on the space levitation image 3E and the space levitation image 3D respectively.

[0362] Here, when user 230 wants to operate the spatial levitation image 3E displayed in front of them from their perspective, their finger will touch the spatial levitation image 3E but not the spatial levitation image 3D. Therefore, if the air operation detection sensor 1351D, which detects the user's operation on the spatial levitation image 3D, does not detect an operation input signal, but the air operation detection sensor 1351E, which detects the user's operation on the spatial levitation image 3E, detects an operation input signal, the air operation detection unit 1350 can determine that "the user is performing a user operation on the spatial levitation image 3E".

[0363] In contrast, if user 230 wants to use their finger to operate the spatial levitation image 3D that appears to be deep in the user's view, the user's finger will touch the spatial levitation image 3D. However, since the spatial levitation image 3E is located in front of the user (nearby), the user's finger or arm is more likely to touch the spatial levitation image 3E.

[0364] Therefore, even if the airborne operation detection sensor 1351E detects an operation input signal, if the airborne operation detection sensor 1351D, which is used to detect user operations on the 3D levitation image, also detects an operation input signal, the airborne operation detection unit 1350 can determine that "the user is performing user operations on the 3D levitation image." In this case, the operation input signal detected by the airborne operation detection sensor 1351E can be ignored.

[0365] Alternatively, the configuration can be such that the xz-direction position of the manipulated icon displayed in the spatial levitation image 3E is offset from the xz-direction position of the manipulated icon displayed in the spatial levitation image 3D, allowing the manipulated icon to be displayed simultaneously in two optical images with different depths. In this case, the aforementioned processing of ignoring the operation input signal detected by the airborne operation detection sensor 1351E can be omitted.

[0366] Based on the above explanation Figure 17C The spatial levitation image display device 1000 is a more compact spatial levitation image display device capable of displaying multiple layers of spatial levitation images with varying degrees of levitation. Furthermore, according to... Figure 17C The spatial levitation image display device 1000 is able to better detect user operations performed on different multi-layer spatial levitation images.

[0367] Next, for those who possess Figure 17A Another example of the space-suspended image display device 1000 with an optical system, using Figure 17D Please provide an explanation.

[0368] Figure 17D This is an example of the structure of a spatial levitation image display device 1000 that displays multi-layered spatial levitation images with different levitation levels. The spatial levitation image display device 1000 shown in the figure is equipped with... Figure 17A The optical system described in the text. Figure 17D An example is an example in which the optical system is configured such that the display device 1 and the mirror reflector 24 are opposite each other in the depth direction (y direction) from the user's perspective, and the spatially suspended image is tilted and floated towards the user's side relative to the vertical direction, i.e., the z direction. Figure 17D in, omit Figure 17A The accompanying drawings show the reference numerals for other elements in the optical system. For example... Figure 17D As shown, the spatial levitation image display device 1000 can display two layers of spatial levitation images, spatial levitation image 3E and spatial levitation image 3D, to the user 230.

[0369] according to Figure 17D The spatial levitation image display device 1000 is a spatial levitation image display device that enables users to view multi-layered spatial levitation images with different levitation amounts from above.

[0370] Next, use Figure 18A An example of a display in a spatial levitation image display device that displays multi-layered spatial levitation images with different levitation amounts will be described. Figure 18A In the diagram, spatial levitation image 3-1 and spatial levitation image 3-2 are multi-layered spatial levitation images with different levitation amounts. For the sake of simplicity, the description of the hardware of the spatial levitation image display device itself is omitted.

[0371] Compared to spatial floating image 3-2, spatial floating image 3-1 appears to be in front of the user. Spatial floating image 3-2 appears to be deeper in the user's view compared to spatial floating image 3-1. Display object 1810 is the object displayed in the display image of spatial floating image 3-1. Display object 1821 is the object displayed in the display image of spatial floating image 3-2. For example, if performing... Figure 18A The spatial levitation image display device is Figure 17D The spatial levitation image display device shown displays multiple layers of images with different levitation levels overlapping in a direction close to the vertical. Thus, Figure 18A In the example, a virtual shadow 1822 is displayed in the display object 1821 of the spatial levitation image 3-2, which appears to be deeper in the user's view compared to the spatial levitation image 3-1. This virtual shadow 1822 appears as if it were caused by the display object 1810 of the spatial levitation image 3-1, which appears to be in front of the user compared to the spatial levitation image 3-2. Here, the virtual shadow can be displayed as black in the corresponding area, or the brightness of the image signal in the corresponding area can be reduced. Additionally, the chroma in the image signal in the corresponding area can also be reduced. These processing methods can be implemented by... Figure 3 The image control unit 1160 and others process the images.

[0372] like Figure 18A As shown, by displaying the virtual shadow of an object in a spatial levitation image on objects in other spatial levitation images located at different depth positions, it is easier to visually recognize the depth relationship between two spatial levitation images that are also spatial levitation images, and can better enhance the user's perception of the actual presence of spatial levitation images.

[0373] Alternatively, it can be configured as follows: Figure 18A As shown, when display objects are shown in multiple spatial floating images with different depths as perceived by the user, the brightness of the display object 1810 in the spatial floating image 3-1, which appears to be in front of the user, is made brighter than the brightness of the display object 1821 in the spatial floating image 3-2, which appears to be deeper in the user's view. This brightness can be changed optically or through image signal processing. By employing this display method, even if the display object in front of the user overlaps with the display object deeper in the user's view, the user can easily identify the brighter object and has difficulty identifying the darker object in the deeper area, thus creating a simulated occlusion. By making the user aware of this simulated occlusion between objects in multiple spatial floating images with different depths, the perceived real presence of the spatial floating images can be significantly enhanced.

[0374] The above explanation Figure 18A For example, the display can be enabled in a way that allows for the use of... Figures 15A to 17A Used in any of the optical systems shown in the spatial levitation image display device.

[0375] Based on the above explanation Figure 18A For example, in a spatial levitation image display device that displays multi-layered spatial levitation images with different levitation amounts, the user's perception of the actual presence of the spatial levitation images can be improved.

[0376] Next, use Figure 18B An example of a display in a spatial levitation image display device that displays multi-layered spatial levitation images with different levitation amounts will be described. Figure 18B The spatial levitation images 3-1 and 3-2 shown are multi-layered spatial levitation images with different levitation levels. From the user's perspective, spatial levitation image 3-1 is displayed in front of them, while spatial levitation image 3-2 is displayed in the depths. Figure 18B For example, the display can be enabled in a way that allows for the use of... Figures 15A to 17A Used in any of the optical systems shown in the spatial levitation image display device. Figure 18B For example, the display can also be achieved Figures 17B to 17D Used in any of the spatial levitation image display devices shown. For example, in... Figure 18B The display example is applied to Figure 17B In the case of the spatial levitation image display device 1000, the spatial levitation image 3-1 corresponds to Figure 17B Spatial levitation image 3E, spatial levitation image 3-2 corresponds to Figure 17B 3D spatial levitation images.

[0377] Figure 18B In the image, object 1850 is the object displayed in the image of the spatial floating image 3-1. Figure 18B In the example, display object 1850 is the display object of the character. Figure 18B In the example, the character is a human character. Display object 1855 and display object 1856 are objects displayed in the display image of the spatial levitation image 3-2. Figure 18B In the example, display object 1855 and display object 1856 are background objects. Figure 18B In the example, the background is a column.

[0378] Right now, Figure 18BIn the example, the character's display object is positioned near the center of the spatial floating image 3-1, which serves as the foreground, while display objects 1855 and 1856, which serve as background objects, are positioned to the left and right, away from the center, in the spatial floating image 3-2. In this example, the main content displayed (the content intended to draw the user's attention) is the character's display object, i.e., display object 1850. Display objects 1855 and 1856 are secondary content, displayed to better highlight the main content, display object 1850.

[0379] Using a spatial levitation image display device capable of displaying multi-layered spatial levitation images with varying degrees of levitation, the advantages of such an object display layout will be explained. For example, in the case of a spatial levitation image display device displaying a single-layered spatial levitation image, when a character is displayed within the spatial levitation image, there are no objects in front of or behind the character as depth references, making it difficult for the user to perceive the depth of the spatial levitation image's display position. In contrast, Figure 18B In the example, a spatial levitation image display device capable of displaying multi-layered spatial levitation images with different levitation amounts is used. The character display object 1850 is displayed near the center in the left and right directions of the spatial levitation image 3-1. Based on this, in the spatial levitation image 3-2 which serves as the background, the display objects 1855 and 1856 of the pillars serving as background objects are arranged in the left and right positions.

[0380] At this point, because of the difference in depth between spatial levitation images 3-1 and 3-2, when the user moves their head to change their viewpoint, due to the principle of motion parallax, the left-right distance and relative position of the column display objects 1855 and 1856 will change relative to the character display object displayed in the center left and right directions. Therefore, the user can more clearly perceive that the character display object displayed in the center left and right directions of spatial levitation image 3-1 is closer to the user than the column display objects 1855 and 1856 displayed on the left and right sides of spatial levitation image 3-2. Here, due to their respective display ranges, these two aerial levitation images, spatial levitation images 3-1 and 3-2, appear to overlap from the user's perspective. At this time, if the image of the spatial floating image 3-2, which appears to be in the depths from the user's perspective, is brighter, and the image of the spatial floating image 3-1, which appears to be in front of the user's perspective, is darker, there is a possibility that the display image of the spatial floating image 3-2, which appears to be in the depths from the user's perspective, is transmitted through the display image of the spatial floating image 3-1, which appears to be in front of the user's perspective, causing the user to be unable to properly perceive the front-back relationship of the displayed objects.

[0381] Therefore, image processing can be performed to adjust the brightness of the display area of ​​the image in the spatial floating image 3-1, which is located in front of the user from their perspective, to be brighter overall, and to adjust the brightness of the display area of ​​the image in the spatial floating image 3-2, which is located in the depths from the user's perspective, to be darker overall. However, depending on the character design of the displayed object in the spatial floating image 3-1, there are cases where it is impossible to brighten the overall brightness of the display area. For example, there may be cases where the character's clothing is dark gray. In the case of such a dark character design, it is preferable to ensure that the displayed object in the spatial floating image 3-2, which is located in the depths from the user's perspective, and the displayed object of the character in the spatial floating image 3-1, which is located in front of the user from their perspective, are not viewed in an overlapping manner.

[0382] Therefore, as Figure 18B As shown, the character display object is positioned near the center in the left-right direction of the spatial floating image 3-1, which serves as the foreground. In the spatial floating image 3-2, which serves as the background, display objects 1855 and 1856 are positioned to the left and right of the character display object in the spatial floating image 3-1 (i.e., near the center in the left-right direction). Thus, even when displaying characters with various character designs, the characters and background objects can be displayed in a way that does not overlap from the user's perspective. The user can more clearly recognize the depth direction display position of the main content, i.e., the character display object 1850, through the aforementioned motion parallax effect, while maintaining the user's depth perception.

[0383] in addition, Figure 18B The motion parallax in the above-mentioned display example is generated based on the actual spatial positions of the spatially suspended images 3-1 and 3-2, and is not simulated motion parallax. This differs from techniques that generate simulated motion parallax through image processing based on the user's viewpoint position. Figure 18B The technology shown in the example does not require image processing based on the user's viewpoint position, thus reducing the processing load. Furthermore, technologies that require image processing based on the user's viewpoint position generally cannot easily support multiple people watching simultaneously, but... Figure 18B The technology shown in the example does not require image processing based on the user's viewpoint position, so it can achieve better motion parallax effects even when multiple users are viewing from different angles.

[0384] Next, use Figure 18C ,right Figure 18B An example of the display image of the display device 1, which is the base image of the spatial levitation image 3-1 and spatial levitation image 3-2 described in the text, will be explained. Figure 18C It is used with Figure 17A optical system Figure 17B The spatial levitation image display device 1000 is used to display... Figure 18B This is an example of the image displayed by the display device 1 in a display case. The display screen 1801 of the display device 1 includes a display area 1501 and a display area 1502. Figure 17B In the spatial levitation image display device 1000, the image displayed in the display area 1502 of the display device 1 is displayed in the air as a spatial levitation image 3E, which corresponds to... Figure 18B Spatial levitation image 3-1.

[0385] Figure 17B In the spatial levitation image display device 1000, the display image of the display area 1501 of the display device 1 is displayed in the air as a spatial levitation image 3D, which corresponds to... Figure 18B Spatial levitation image 3-2. Figure 18B The display image in display area 1502 shown in the spatial levitation image 3-1 is related to... Figure 18B Compared to the image displayed in display area 1501 in the spatial levitation image 3-2, it is displayed on the side in front of the user. Additionally, when using... Figures 15A-16B In the case of the optical system, the front-to-back relationship of the spatial levitation image 3D and spatial levitation image 3E in terms of depth direction from the user's perspective is opposite, so the display image of display area 1501 is displayed in front of the user in the air compared to the display image of display area 1502.

[0386] like Figure 18C As shown, in the spatial levitation image display device 1000 of this embodiment, an image source that displays two images at different depths in the air can be displayed using a single hardware device, namely the display device 1. Although the images are displayed in the display areas 1501 and 1502 of the display device 1, the images of each frame of the two images are stored in the frame memory of the display device 1 as if contained within a single image. Therefore, compared to a structure in which two images are displayed using different display devices, it is more preferable that a complex synchronization system or the like is not required to synchronize the two images. Compared to a structure in which two images are displayed using different display devices, it is more cost-effective to implement various processing-related hardware such as two sets of display memory.

[0387] Here, Figure 18C In one example of the image displayed by the display device 1, a gap 1807 is provided between the display area 1501 and the display area 1502. In the area of ​​this gap 1807, the display device 1 fixes the image to black. The reason for this is explained below. Figures 15A-17AAs described above, in any of the optical systems, a light-shielding plate is provided between display area 1501 and display area 1502 on the emitting surface of the display screen of display device 1. This light-shielding plate is provided to prevent the image light emitted from display area 1501 and the image light emitted from display area 1502 from mixing in their respective optical paths as much as possible.

[0388] More preferably, a gap 1807 is provided between display area 1501 and display area 1502, and the width of the gap 1807 is greater than the thickness of the light shield provided between display area 1501 and display area 1502 on the emitting surface of the display screen of display device 1. Therefore, the image light emitted from display area 1501 and the image light emitted from display area 1502 are not blocked by the light shield, and their mixing in each other's optical paths is minimized. Furthermore, the area of ​​gap 1807 has been described as displaying the image as fixed in black, but it can also be expressed as a non-display area where no content is displayed.

[0389] Here, use Figure 3 Structural description of the spatial levitation image display device 1000 regarding its implementation Figure 18C The first processing example is a case of processing the display example. The first processing example is an example of reproducing and displaying the image to be displayed in the display area 1501 and the image to be displayed in the display area 1502 from the storage unit 1170.

[0390] Specifically, the image information of the character of the display object 1850 and the background image information of the display objects 1855 and 1856 as background objects are accumulated in the storage unit 1170. The image control unit 1160 reproduces the image information of the character of the display object 1850 and configures the reproduced image information in accordance with the image information of the display object 1850. Figure 18C The position corresponding to the display area 1502 of the display device 1. The image control unit 1160 can also control the reproduction of background image information including display object 1855 and display object 1856 accumulated in the storage unit 1170, and display it. Figure 18C In the display area 1501 of the display device 1.

[0391] Additionally, using Figure 3 Structural description of the spatial levitation image display device 1000 regarding its implementation Figure 18C The second processing example is a case where the content creator has prior knowledge of the processing of the display example. Figure 18CThe layout of the screen 1801 of the display device 1 shown, including the overall screen, display area 1501 and display area 1502, is as follows: the content creator creates image content corresponding to the screen 1801 including display area 1501 and display area 1502, accumulates the content in the storage unit 1170, and the image control unit 1160 controls the reproduction of the image of the content and displays it on the entire screen of the screen 1801 of the display device 1.

[0392] The image content corresponds to the display screen 1801. The position corresponding to the display area 1502 includes the image of the display object 1850 as a character, and the positions corresponding to the display area 1501 include the images of the display objects 1855 and 1856 as background objects. The image content is already present in the storage unit 1170 at the point in time it is accumulated. Figure 18C The layout of the screen 1801, display area 1501, and display area 1502 corresponds, so the image control unit 1160 only needs to control the reproduction of the image of the content and display it on the entire screen 1801 of the display device 1. When displaying the content, there is no need to perform complex image overlay processing, which can reduce the amount of processing.

[0393] Additionally, using Figure 3 Structural description of the spatial levitation image display device 1000 regarding its implementation Figure 18C The third processing example is the display example. In the third processing example, the content creator has prior knowledge of... Figure 18C The layout of the screen 1801 of the display device 1 shown, the display area 1501 and the display area 1502, the content creator creates image content corresponding to the screen 1801 including the display area 1501 and the display area 1502, and accumulates the content in an external device different from the spatial floating image display device 1000.

[0394] In order to enable the spatially suspended image display device 1000 Figure 3 The image signal input unit 1131 receives an image output signal from the aforementioned external device, which is pre-connected to the spatial levitation image display device 1000. The external device outputs an image signal containing image content corresponding to the display screen 1801, which includes display areas 1501 and 1502, and inputs it to the image signal input unit 1131 of the spatial levitation image display device 1000. The image control unit 1160 controls the reproduction of the image signal containing the image content input to the image signal input unit 1131 and displays it on the display screen 1801 of the display device 1.

[0395] Image content and about realization Figure 18CThe second processing example is the same as the display example, so repeated explanations are omitted. The image of the content is already synchronized with the image signal input unit 1131 at the point when it is input. Figure 18C The layout of the screen 1801, display area 1501, and display area 1502 corresponds, so the image control unit 1160 only needs to control the reproduction of the image of the content and display it on the entire screen 1801 of the display device 1. When displaying the content, there is no need to perform complex image overlay processing, which can reduce the amount of processing.

[0396] Additionally, using Figure 3 Structural description of the spatial levitation image display device 1000 regarding its implementation Figure 18C The fourth processing example is an example of using an image generation program to generate an image to be displayed in display area 1501 and an image to be displayed in display area 1502, respectively, based on a 3D model rendering.

[0397] Specifically, firstly, an image generation program is stored in storage unit 1170. This program can generate rendered images of 3D models of characters corresponding to display object 1850, and can also generate rendered images of 3D models of background objects corresponding to display objects 1855 and 1856. Control unit 1110 reads the image generation program from storage unit 1170 and loads it into memory 1109. Control unit 1110 executes the image generation program loaded into memory 1109, which renders the 3D models of characters to generate an image of display object 1850.

[0398] The image control unit 1160 controls the display of the generated image of the display object 1850. Figure 18C The image generation program renders the 3D model of the background object in parallel, generating images of display object 1855 and display object 1856. The image control unit 1160 can control the display of the generated images of display object 1855 and display object 1856. Figure 18C In display area 1501.

[0399] In addition, the main content plays a role in Figure 18B The example uses a human character, but it could also be an animal character, a robot character, or a so-called virtual avatar used in virtual space. Here, the display object 1850 can use a character image rendered based on a 3D model. Alternatively, it can use a 2D animated character. Or it can use a real-world image of a person, etc., as the character image.

[0400] Additionally, display objects 1855 and 1856, which are secondary content, are... Figure 18B In the example, it could be an object representing a pillar, but it could also be a virtual frame object, or furniture or equipment placed in the space where the main character is located. Any background object located behind the character in the space will suffice.

[0401] Based on the above explanation Figure 18B and Figure 18C For example, in a spatial levitation image display device that displays multi-layered spatial levitation images with different levitation amounts, it is possible to achieve a display in which the user can more clearly recognize the display position in the depth direction for the display object such as the main content, and at the same time, the user's perception of that depth can be better maintained.

[0402] Next, use Figure 18D Another example of a display in a spatial levitation image display device that displays multi-layered spatial levitation images with different levitation amounts will be described. Figure 18D The spatial levitation images 3-1 and 3-2 shown are multi-layered spatial levitation images with different levitation levels. From the user's perspective, spatial levitation image 3-1 is displayed in front of them, while spatial levitation image 3-2 is displayed in the depths.

[0403] Figure 18D For example, the display can be enabled in a way that allows for the use of... Figures 15A to 17A Used in any of the optical systems shown in the spatial levitation image display device. Figure 18D For example, the display can also be achieved Figures 17B to 17D Used in any of the spatial levitation image display devices shown. For example, in... Figure 18D The display example is applied to Figure 17B In the case of the spatial levitation image display device 1000, the spatial levitation image 3-1 corresponds to Figure 17B Spatial levitation image 3E, spatial levitation image 3-2 corresponds to Figure 17B 3D spatial levitation images.

[0404] Figure 18D In the image, object 1851 is the object displayed in the image of the spatial levitation image 3-2. Figure 18D In the example, display object 1851 is the display object of the character. Figure 18D In the example, the character is a person. Figure 18D In the example, display object 1857 and display object 1858 are objects displayed in the display image of spatial levitation image 3-1. Figure 18D In the example, display object 1857 and display object 1858 are foreground objects. Figure 18DIn the example, the foreground object refers to the object that should be displayed in front of the main content in the space (from the user's perspective). Figure 18D In the example, the foreground is characters (text).

[0405] Right now, Figure 18D In this example, characters are displayed as secondary content in the foreground of display object 1851, which is the main content being displayed. For instance, an animated image of a character singing is shown on display object 1851, with the lyrics of the song displayed in the foreground on display objects 1857 and 1858. In this diagram, display object 1857 shows an example of horizontally arranged characters, and display object 1858 shows an example of vertically arranged characters.

[0406] Figure 18D In the example, the lyrics of the song being sung by the character in the suspended spatial image 3-2, namely display objects 1857 and 1858, are displayed in the foreground suspended spatial image 3-1 in sync with the character's singing animation. That is, as the singing progresses, the displayed lyrics change in conjunction with it. The display position, character direction, size, font, etc., of the lyrics can be displayed in various ways. The characters can scroll in response to the progress of the singing, or the position or size can be changed to switch a few characters at a time.

[0407] and Figure 18B The display examples differ, and there is no particular need to avoid overlap between the character display object 1851 and the character display objects 1857 and 1858. As long as it is effective in terms of display effect, it can be done as follows. Figure 18D The images are shown in an overlapping manner when viewed from the front.

[0408] Furthermore, in general, there are techniques for overlaying images and character information in flat 2D displays. However, when images and character information are overlaid on a flat 2D display without special processing, the positional relationship between the image and character information does not produce motion parallax even if the user changes their viewpoint. Therefore, even if the character information is overlaid in front of the user, it appears to be overlapping on the same plane as the plane containing the image, and the user cannot easily perceive that the character information is displayed at a different depth compared to the image.

[0409] In addition, at this time, in a flat 2D display, for an image with superimposed character information, even if the user changes the viewpoint, the positional relationship between the image and the character information will not produce motion parallax, so it is easy to recognize that it is a flat image, and the user cannot easily view the image in three dimensions.

[0410] In contrast, Figure 18DIn the example, spatial levitation images 3-1 and 3-2 are real images with depth differences. Therefore, when the user moves their head to change their viewpoint, due to the principle of motion parallax, the distance and relative position of the character display objects 1857 and 1858 will change relative to the position of the character display object 1851. Thus, the user can easily perceive that the character display objects 1857 and 1858 are displayed in front of them compared to the character display object 1851.

[0411] At this point, it is easy to recognize that the display objects 1857 and 1858 of the characters displayed in the spatial levitation image 3-2 are not on the same plane as the display object 1851 of the character displayed in the spatial levitation image 3-1. Therefore, the display of the display object 1851 of the character displayed in the spatial levitation image 3-1 is not limited by the planar perception that it is on the same plane as the display objects 1857 and 1858 of the characters, which is more preferable.

[0412] in addition, Figure 18D The motion parallax in the above-mentioned display example is generated based on the actual spatial positions of the spatially suspended images 3-1 and 3-2, and is not simulated motion parallax. This differs from techniques that generate simulated motion parallax through image processing based on the user's viewpoint position. Figure 18D The technology shown in the example does not require image processing based on the user's viewpoint position, thus reducing the processing load. Furthermore, technologies that require image processing based on the user's viewpoint position generally cannot easily support multiple people watching simultaneously, but... Figure 18D The technology shown in the example does not require image processing based on the user's viewpoint position, so it can achieve better motion parallax effects even when multiple users are viewing from different angles.

[0413] in addition, Figure 18D The example shows both display object 1857 and display object 1858 displayed simultaneously, but they are each examples of character display forms and do not necessarily have to be displayed at the same time. There may also be times when neither is displayed.

[0414] Next, use Figure 18E ,right Figure 18D An example of the display image of the display device 1, which is the base image of the spatial levitation image 3-1 and spatial levitation image 3-2 described in the text, will be explained. Figure 18E It is used with Figure 17A optical system Figure 17B The spatial levitation image display device 1000 is used to display... Figure 18D This is an example of the display image displayed by display device 1 in the case of a display example.

[0415] The display screen 1801 of the display device 1 includes a display area 1501 and a display area 1502. Figure 17B In the spatial levitation image display device 1000, the image displayed in the display area 1502 of the display device 1 is displayed in the air as a spatial levitation image 3E, which corresponds to... Figure 18D Spatial levitation image 3-1. Figure 17B In the spatial levitation image display device 1000, the display image of the display area 1501 of the display device 1 is displayed in the air as a spatial levitation image 3D, which corresponds to... Figure 18D Spatial levitation image 3-2. Figure 18D The display image in display area 1502 shown in the spatial levitation image 3-1 is related to... Figure 18D Compared to the display image in display area 1501 shown in the spatial levitation image 3-2, it is displayed in front of the user.

[0416] In addition, in use Figures 15A-16B In the case of the optical system, the front-to-back relationship of the spatial levitation image 3D and spatial levitation image 3E in terms of depth direction from the user's perspective is opposite, so the display image of display area 1501 is displayed in front of the user in the air compared to the display image of display area 1502.

[0417] Here, as Figure 18E As shown in the example, an image source capable of displaying two images at different depth positions in the air can be displayed using a single hardware device, namely display device 1. This advantage is similar to... Figure 18C The advantages described in the previous section are the same, so repeated explanations are omitted. Additionally, as... Figure 18E As shown in the example, the advantages of setting a gap 1807 between display area 1501 and display area 1502 are... Figure 18C The advantages described in the previous section are the same, so repeated explanations are omitted.

[0418] Here, use Figure 3 Structural description of the spatial levitation image display device 1000 regarding its implementation Figure 18E The first processing example is a case of processing the display example. The first processing example is an example of reproducing and displaying the image to be displayed in the display area 1501 and the image to be displayed in the display area 1502 from the storage unit 1170.

[0419] The image control unit 1160 accumulates additional information, such as the image information of the character singing on display object 1851, the character information of the lyrics, and the display timing information of the character information, in the storage unit 1170. The image control unit 1160 then reproduces the image information of the character singing on display object 1851 and configures the reproduced image information in accordance with... Figure 18EThe position corresponding to the display area 1501 of the display device 1.

[0420] The image control unit 1160 can also control the reproduction of character information and additional information accumulated in the storage unit 1170, and use display timing information to synchronize the display with the aforementioned image information. Figure 18E The information is displayed in the display area 1502 of the display device 1. At this time, if the additional information includes the display position, size, font, and display color of the character information, the display position, size, font, and display color of the character information can be determined and displayed based on this information.

[0421] Additionally, using Figure 3 Structural description of the spatial levitation image display device 1000 regarding its implementation Figure 18E The second processing example is a case where the content creator has prior knowledge of the processing of the display example. Figure 18E The layout of the screen 1801 of the display device 1 shown, including the overall screen, display area 1501 and display area 1502, is as follows: the content creator creates image content corresponding to the screen 1801 including display area 1501 and display area 1502, accumulates the content in the storage unit 1170, and the image control unit 1160 controls the reproduction of the image of the content and displays it on the entire screen of the screen 1801 of the display device 1.

[0422] The image content corresponds to the display screen 1801. The position corresponding to display area 1501 includes the image of the singing character, i.e., display object 1851. The position corresponding to display area 1502 includes the images of the lyrics characters, i.e., display objects 1857 and 1858. The image content is already associated with the display screen 1801 at the point in time it is accumulated in storage unit 1170. Figure 18E The layout of the screen 1801, display area 1501, and display area 1502 corresponds, so the image control unit 1160 only needs to control the reproduction of the image of the content and display it on the entire screen 1801 of the display device 1. When displaying the content, there is no need to perform complex image overlay processing, which can reduce the amount of processing.

[0423] Additionally, using Figure 3 Structural description of the spatial levitation image display device 1000 regarding its implementation Figure 18E The third processing example is the display example. In the third processing example, the content creator has prior knowledge of... Figure 18EThe layout of the screen 1801 of the display device 1 shown, the display area 1501 and the display area 1502, the content creator creates image content corresponding to the screen 1801 including the display area 1501 and the display area 1502, and accumulates the content in an external device different from the spatial floating image display device 1000.

[0424] In order to enable the spatially suspended image display device 1000 Figure 3 The image signal input unit 1131 receives an image output signal from the aforementioned external device, which is pre-connected to the spatial levitation image display device 1000. The external device outputs an image signal containing image content corresponding to the display screen 1801, which includes display areas 1501 and 1502, and inputs it to the image signal input unit 1131 of the spatial levitation image display device 1000. The image control unit 1160 controls the reproduction of the image signal containing the image content input to the image signal input unit 1131 and displays it on the display screen 1801 of the display device 1.

[0425] Image content and about realization Figure 18E The second processing example is the same as the display example, so repeated explanations are omitted. The image of the content is already synchronized with the image signal input unit 1131 at the point when it is input. Figure 18E The layout of the screen 1801, display area 1501, and display area 1502 corresponds, so the image control unit 1160 only needs to control the reproduction of the image of the content and display it on the entire screen 1801 of the display device 1. When displaying the content, there is no need to perform complex image overlay processing, which can reduce the amount of processing.

[0426] Additionally, using Figure 3 Structural description of the spatial levitation image display device 1000 regarding its implementation Figure 18E The fourth processing example is an example of using an image generation program to generate an image to be displayed in display area 1501 and an image to be displayed in display area 1502, respectively, based on a 3D model rendering.

[0427] Specifically, firstly, an image generation program is stored in storage unit 1170. This program can generate a rendered image of the 3D model of the character corresponding to display object 1851, and can also generate a rendered image of the model in 3D space of the character information corresponding to display objects 1857 and 1858. Control unit 1110 reads the image generation program from storage unit 1170 and loads it into memory 1109. Control unit 1110 executes the image generation program loaded into memory 1109, which renders the 3D model of the character performing the currently singing animation to generate an image of display object 1851.

[0428] The image control unit 1160 controls the display of the generated image of the display object 1851. Figure 18E The image generation program renders the character information model in 3D space in parallel, generating images of display objects 1857 and 1858. The image control unit 1160 can control the display of the generated images of display objects 1857 and 1858. Figure 18E In the display area 1502.

[0429] Additionally, the main character displayed as object 1851 can be a character image rendered from a 3D model. Alternatively, a 2D animated character can be used. Or, a live-action image of a person can be used as the character image. The character displayed as object 1851 can also be a promotional video of a character or person singing music.

[0430] Additionally, display objects 1857 and 1858, which are secondary content, are... Figure 18D For example, a display object could be the characters of the lyrics to a song a character is singing, but it's not limited to that. It could also be a display object for so-called special effects images displayed in front of the character. Specific examples of special effects images include those showing flashing stars, lightning, rain, snow, or falling petals—any display object that represents a special effect displayed in front of the character. These special effects images can also be displayed in conjunction with the animation of the main character.

[0431] When special effects images are displayed in display objects 1857 and 1858, which are secondary content, display objects 1857 and 1858 are also displayed as spatially suspended images with a different depth compared to display object 1851, which is the main content, thus creating motion parallax. This results in the effect that even if the display objects of the special effects images overlap in front of the display objects of the characters, it is less likely to be limited by the planar recognition of the display objects of the characters, etc.

[0432] Based on the above explanation Figure 18D and Figure 18E In a spatial levitation image display device that displays multi-layered spatial levitation images with different levitation amounts, for the main content such as characters, character information and special effects images, which are secondary content, can be displayed in conjunction at different display positions in the depth direction. Therefore, even if the character display objects and special effects image display objects are displayed in overlapping positions relative to the character display objects, it is less likely to be limited by the planar recognition of the character display objects, which is preferable.

[0433] <Example 4>

[0434] As Embodiment 4 of the present invention, a structural example of a spatial levitation image display device will be described. Embodiment 4 is similar to Embodiment 3 described above. Figures 15A-18E Similarly, it is a spatial levitation image display device capable of displaying multi-layered (especially two-layered) spatial levitation images of varying amounts raised from the optical system. In this embodiment, the differences from Embodiment 3 will be mainly explained.

[0435] The spatial levitation image display device of Embodiment 4 forms two layers of spatial levitation images 3 in the depth direction as viewed from the user. As this spatial levitation image display device 1000, for example, an application... Figures 15A-18E In Figure 17C The structure will be explained below. In some cases, [the following will be...] Figure 17C From the user's viewpoint 230, the spatial floating image 3E formed in the depth direction, i.e., the y-direction, on the front / foreground is denoted as the first layer or the first aerial floating image, and the spatial floating image 3D formed on the rear / background is denoted as the second layer or the second aerial floating image.

[0436] [First Function]

[0437] Figure 19A This is a schematic perspective view illustrating a display example of two layers of suspended spatial images 3 (3D, 3E) in Embodiment 4. Suspended spatial image 3E is the first layer on the front side, and suspended spatial image 3D is the second layer on the rear side. The coordinate system (x, y, z) shown in the illustration is... Figure 17C The coordinate system is consistent. Embodiment 4 has the following first function, wherein the display control can use two layers of spatially suspended images 3 (3D, 3E) to emphasize the sense of distance of the graphical user interface (GUI) such as buttons.

[0438] The following technical problem exists here. Because the spatial levitation image display device forms a real image in the air, there is no tactile feedback when the user touches the spatial levitation image. For example, there are cases where a GUI such as buttons is displayed in the spatial levitation image, and the user's finger (or, for example, a handheld stick, etc., can be used to perform airborne / touch operations on the button, etc.) via the aforementioned sensors, etc. Figure 3 The system detects the operation using an in-flight operation detection sensor 1351 and an in-flight operation detection unit 1350, etc. In this situation, even if the user intends to press the button on the levitating image with their finger, they do not experience any tactile feedback from the button. Therefore, it is difficult for the user to recognize or determine whether they successfully touched the button or whether the operation was correct. In other words, there is a problem with the tactile feedback regarding in-flight operation of the levitating image.

[0439] Regarding this problem, one possible solution is to emit a clicking sound when the button or other element of the suspended spatial image is touched. However, this solution is ineffective in noisy environments. Example 4 provides the following solution to the aforementioned technical problem.

[0440] Example 4: Spatial Suspended Image Display Device Figure 19A As shown, in the two-layer spatial levitation image 3 (3E, 3D), the button icon 1901E, which serves as a GUI and specifically a button, is first displayed in the front spatial levitation image 3E. At this point, the button icon 1901D is not displayed in the rear spatial levitation image 3D. In this example, the button icon 1901E displays an image of the text "Push" within a square area.

[0441] The user touches the button icon 1901E with their hand UH (e.g., finger UF). This touch / air operation refers to moving and positioning the finger UF by pressing the button icon 1901E deeper in the depth direction (y-direction). The spatial levitation image display device of Embodiment 4 uses the aforementioned sensors to detect such touch operations on the button icon 1901E. That is, the spatial levitation image display device detects whether a finger UF exists in the area of ​​the button icon 1901E within the spatial levitation image 3E corresponding to the xz plane.

[0442] When such a touch operation is detected, the spatial levitation image display device of Embodiment 4 controls the display of the two layers of spatial levitation images 3 (3D, 3E) by reducing the brightness of the button icon 1901E of the front spatial levitation image 3E and increasing the brightness of the button icon 1901D of the rear spatial levitation image 3D. The button icon 1901D on the deeper side (i.e., the rear side) is the same image as the button icon 1901E on the front side in terms of color, shape, etc., and is formed at a corresponding position in the front-back direction (y direction). Thus, from the user's perspective, it can be perceived that the button icons (1901E, 1901D) are pressed deeper from the front as a touch operation is performed.

[0443] Furthermore, after the aforementioned operation and display, for example, after a certain period of time, the spatial levitation image display device of Embodiment 4 restores the brightness of each button icon (1901E, 1901D) to its initial value. That is, the spatial levitation image display device controls the display of the two layers of spatial levitation images 3 (3D, 3E) by increasing the brightness of button icon 1901E of the front spatial levitation image 3E to its initial value while simultaneously decreasing the brightness of button icon 1901D of the rear spatial levitation image 3D to its initial value. Thus, from the user's perspective, a visual effect can be perceived as the button icons (1901E, 1901D) recovering from a depth towards the front.

[0444] Example 4 clearly shows the situation where the button of the spatial levitation image 3 is pressed through this processing, thus improving the user's sense of operation.

[0445] Figure 19B express Figure 19A An example of display control, illustrating the brightness change of a button icon when a user touches it. Figure 19B The horizontal axis of the graph represents time, and the vertical axis represents the display brightness of the button icons before and after.

[0446] Using sensors, etc. (e.g.) Figure 17C The aerial operation detection sensor 1351E, etc., detects that the user has touched the button icon 1901E on the front side (time point t1). Then, the spatial levitation image display device (e.g., Figure 3 The image control unit 1160 gradually reduces the brightness of the button icon 1901E on the front-side suspended spatial image 3E, bringing it to a predetermined reduced brightness, for example, a non-display state (time point t2). Then, the non-display state of the button icon 1901E is maintained for a predetermined time. Afterwards (time point t3), the suspended spatial image display device gradually increases the brightness of the button icon 1901E again, restoring it to its original initial brightness value (time point t4).

[0447] In conjunction with the display control of the front button icon 1901E, the display control of the rear button icon 1901D is performed as shown in the figure. The rear button icon 1901D, unlike the previous one, gradually increases in brightness from an initial predetermined brightness (e.g., a non-display state) (time point t1) until it is displayed at a predetermined brightness (time point t2). Then, the display state of button icon 1901D is maintained for a predetermined time (time point t3). Afterward, the brightness of button icon 1901D is gradually decreased again, returning to its original initial brightness value, i.e., the non-display state (time point t4).

[0448] Time 1901DT is the display time of the rear button icon 1901D. As a function of this button icon, it is in an OFF state when the front button icon 1901E is displayed, and turns ON when the front button icon 1901E is touched (pressed). The spatial levitation image display device can perform pre-defined processing in response to this ON operation.

[0449] In this example, the brightness adjustment of the button icons is controlled linearly as shown in the figure, but this is not a limitation. Additionally, in this example, the button icons use a GUI that automatically returns to their original front position (OFF state) after a predetermined time following an ON operation, but this is not a limitation. In other examples, ON / OFF toggle type button icons can also be used. In this case, when the front button icon 1901E is touched, it switches to the OFF state while the rear button icon 1901D remains displayed. Then, upon detecting a touch operation on the rear button icon 1901D, it switches to the ON state, displaying the front button icon 1901E.

[0450] Furthermore, as an example of sensor structure, the airborne operation detection sensor 1351 senses, for example, two-layer spatial suspended images 3 (3D, 3E) with... Figure 17C The same sensing is performed by the 1351D and 1351E airborne operation detection sensors. Figure 19BThe outline is illustrated in the dialog box. In this example sensor structure, for the first layer of spatial levitation image 3E at the front, an air operation detection sensor 1351E, positioned slightly upwards, illuminates light downwards in the z-direction (e.g., infrared light, non-display light), covering the plane (xz plane) of the spatial levitation image 3E. Similarly, for the second layer of spatial levitation image 3D at the rear, an air operation detection sensor 1351D, positioned slightly upwards, illuminates light downwards in the z-direction, covering the plane (xz plane) of the spatial levitation image 3D. With such a sensor structure, it is possible to detect at least whether a finger touches the plane of the spatial levitation image 3E, the spatial levitation image 3D, and the position of the finger within the plane. The sensor structure is not limited to this and may also include a camera unit, etc.

[0451] Figure 19C Indicates viewing from the front Figure 19A Examples of display regarding the dimensions of each image in the case of button icons 1901D and 1901E. (A) shows the display of the rear button icon 1901D on the xy plane. (B) shows the display of the front button icon 1901E on the xy plane. (A) and (B) correspond in terms of size.

[0452] To further emphasize, Figure 19A The sense of distance in the button icons displayed in the two-layer spatial floating image 3 (3D, 3E) shown can be achieved by using... Figure 19C As shown. Figure 19C In this example, if the size of the front button icon 1901E is X in the horizontal direction (x-direction) and Y in the vertical direction (z-direction), then the size of the rear button icon 1901D is reduced accordingly for display. In this example, the size of the rear button icon 1901D is reduced to 0.9 times, with a horizontal dimension of 0.9X and a vertical dimension of 0.9Y.

[0453] Through such size display control, it is possible to emphasize, from the user's perspective, [the following is an example of how to do something]. Figure 19A The perceived distance of button icons can help improve the user experience.

[0454] in addition, Figure 19D Other examples of displays that further emphasize the sense of distance. Figure 19D In the diagram, the horizontal axis represents the front and rear button icons, and the vertical axis represents brightness, luminance, or chroma. In this example, the luminance of the display color of the front button icon 1901E is used as a reference, and the luminance of the rear button icon 1901D is reduced. Specifically, it is preferable to reduce the luminance by a factor of 0.4 to 0.7. This example shows the case where the luminance of the front button icon 1901E is reduced to 0.5 times that of the rear button icon 1901D. Through such display control of luminance, etc., the user can further emphasize... Figure 19A The perceived distance of button icons can help improve the user experience.

[0455] [Second Function]

[0456] As another function in Example 4 (referred to as the second function). Figure 19E This indicates the function of moving an object picture between two suspended spatial images 3 (3D, 3E) via user operation. (A) indicates that the virtual image 2001E of the character is displayed as the object picture in the front suspended spatial image 3E. At this time, the character image 2001D is not displayed in the rear suspended spatial image 3D. (B) indicates that the virtual image 2001D of the character is displayed as the object picture in the rear suspended spatial image 3D. At this time, the character image 2001E is not displayed in the front suspended spatial image 3E. The character image 2001E in the front and the character image 2001D in the rear are images of the same virtual image.

[0457] Here, when processing multi-layered spatial levitation images 3 in the depth direction, or when processing touch operations in the depth direction, the following technical problems exist. In the spatial levitation image display device, the aforementioned sensors (air operation detection sensor 1351 and air operation detection unit 1350) can detect touches within a two-dimensional plane corresponding to the spatial levitation image 3. However, depending on the implementation, there are cases where it is difficult to use this sensor to detect touches in the depth direction (in other words, the front-to-back direction) of the two-dimensional plane; for example, it is difficult to detect the position and distance of a finger in front of or behind the two-dimensional plane. In this case, it is difficult to perform operations and detection such as the movement of the target image in the depth direction (e.g., movement across two spatial levitation images 3 (3D, 3E)).

[0458] Therefore, as a solution to the aforementioned problem, Embodiment 4 has the following function. In this function, specific touch / air operations for the two layers of suspended spatial images 3 (3D, 3E) are assigned to operations that move the object image between the two layers of suspended spatial images 3 (3D, 3E) in the depth direction (sometimes referred to as depth movement operations, etc.). In other words, specific touch operations on the plane of the suspended spatial images 3 are predefined and set as predetermined depth movement operations. The suspended spatial image display device detects this specific operation, i.e., the depth movement operation, using the aforementioned sensors. When a depth movement operation is detected, the suspended spatial image display device controls the display of the suspended spatial images 3 (3D, 3E) in a manner that moves the object image between the two layers of suspended spatial images 3 (3D, 3E) in the depth direction. Figure 19E Provide a specific example illustrating this function.

[0459] As shown in (A), initially, the image 2001E of a virtual character, i.e., a virtual avatar, is displayed in the front spatial levitation image 3E. In this state, the user uses their finger to perform a predetermined depth movement operation with the image 2001E of the character as the target. This predetermined depth movement operation is, for example, a predetermined touch operation performed on a portion of the image 2001E of the character in the front spatial levitation image 3E (described later). The spatial levitation image display device uses sensors or the like to detect the touch operation corresponding to the predetermined depth movement operation. Accordingly, the spatial levitation image display device controls the display of the two layers of spatial levitation images 3 (3D, 3E) in such a way that the image 2001E of the character displayed in the front spatial levitation image 3E is moved to be displayed in the rear spatial levitation image 3D. Thus, as indicated by the arrow, the image 2001D of the character is displayed in the rear spatial levitation image 3D, becoming the same state as in (B).

[0460] Conversely, as shown in (B), with the character's image 2001D displayed in the rear spatial levitation image 3D, the user targets the character's image 2001D and performs a predetermined depth movement operation with their finger. This predetermined depth movement operation is, for example, a predetermined touch operation performed on a portion of the character's image 2001D in the rear spatial levitation image 3D (described later). The spatial levitation image display device detects this predetermined depth movement operation using sensors or the like. Accordingly, based on this detection, the spatial levitation image display device controls the display of the two layers of spatial levitation images 3 (3D, 3E) in a manner that moves the character's image 2001D displayed in the rear spatial levitation image 3D to be displayed in the front spatial levitation image 3E. Thus, as indicated by the arrow, the character's image 2001E is displayed in the front spatial levitation image 3E, becoming the same state as in (A).

[0461] As described above, through the prescribed depth movement operation and detection, it is possible to easily move object images such as virtual figures between the preceding and following spatially suspended images 3 (3D, 3E). From the user's perspective, a visual effect of the virtual figure moving in the depth direction can be obtained.

[0462] Figure 19EIn the example shown, the display state (e.g., posture) of the character image 2001E in the front spatial levitation image 3E is the same as the display state (e.g., posture) of the same character image 2001D in the rear spatial levitation image 3D, but this is not a limitation. The display state of the characters (2001E, 2001D) can also be changed as the depth direction moves. As the depth direction moves, for example, animation-based motion can be used to add the effect of the character image 2001E jumping and moving from the front spatial levitation image 3E to the rear spatial levitation image 2001D. For example, the display can be controlled so that the character image 2001E in the front spatial levitation image 3E changes from a forward-facing posture to a backflip jump, and then lands in the character image 2001D in the rear spatial levitation image 3D and changes to a forward-facing posture.

[0463] Figure 19E The depth movement operation in case (A) and the depth movement operation in case (B) can be considered as the same touch operation or as different touch operations, depending on the rules. For example, when the character's image 2001D is displayed in the rear suspended image 3D as shown in case (B), and a specified depth movement operation is detected for the front suspended image 3E, the character's image (2001D, 2001E) can be moved from the rear suspended image 3D to the front suspended image 3E. Furthermore, when the character's image 2001E is displayed in the front suspended image 3E as shown in case (A), and a specified depth movement operation is detected for the rear suspended image 3D, the character's image (2001D, 2001E) can be moved from the front suspended image 3E to the rear suspended image 3D.

[0464] in addition, Figure 19E In the example, it is assumed that a depth-shifting operation targeting the images (2001E, 2001D) of the target character is detected, but it is not limited to this. If a specified depth-shifting operation targeting a region other than a portion of the images (2001E, 2001D) of the target character is detected within the plane of the spatially suspended images 3 (3D, 3E), depth-shifting processing can also be performed targeting the images (2001E, 2001D) of the character.

[0465] Figure 19F Indicates about Figure 19E The processing flow for in-depth movement operations. Spatial levitation image display device. Figure 3 The image control unit 1160 performs as follows Figure 19FThe process is shown. In step S19F1, the space-based levitation image display device detects, based on the airborne operation detection sensor 1351 and the airborne operation detection unit 1350, the following... Figure 19E The image shows a user's touch operation on two layers of suspended spatial images 3 (3D, 3E). In step S19F2, the suspended spatial image display device determines whether the touch operation is a defined depth-shifting operation. If it is a defined depth-shifting operation ("Yes"), proceed to step S19F3; otherwise, proceed to step S19F4. In step S19F3, the suspended spatial image display device performs a process (depth-shifting process) that moves the target image of the object subject to the defined depth-shifting operation between the two layers of suspended spatial images 3 (3D, 3E). The content of this process is as follows: Figure 19E As shown. In step S19F4, the spatial levitation image display device performs processing within the same screen, that is, the prescribed processing corresponding to the touch operation within the plane of the 1st layer spatial levitation image 3 (3D or 3E) that is subjected to the touch operation.

[0466] Figure 19G The table shows about Figure 19E Several examples of the specified depth-of-motion operation are provided. Examples of touch operations assigned to the depth-of-motion operation include, as shown in the table, long press, swipe, multi-touch, double-tap, etc. At least one of these operations is assigned to the depth-of-motion operation. The software and hardware for this depth-of-motion operation are pre-implemented / installed in the spatial levitation image display device.

[0467] The depth-of-motion operation 19G1 is a long-press operation on a plane of a spatially suspended image 3. It targets the same area within that plane and maintains a state of touch (i.e., positioned in the air) with a finger or other object for a certain period of time. For example, in... Figure 19E In (A), the operation is to press and hold the part of the image 2001E of the character in the space-floating image 3E on the front side with your finger.

[0468] The depth movement operation 19G2 is a stroking operation on a plane of a spatially suspended image 3. It is an operation that targets the same part within the plane and, while maintaining contact with a finger or other object, rapidly slides the finger or other object along the plane. The direction of this stroking is not limited.

[0469] The depth movement operation 19G3 is a multi-touch operation on a plane of a spatial floating image 3. It is an operation that uses two or more fingers to touch the same part of the plane at the same time.

[0470] The depth movement operation 19G4 is a double-click operation on a plane of a spatial suspended image 3, which is an operation that involves two consecutive taps on the same part within that plane as the target part.

[0471] Not limited to this example, a specific touch operation on the plane of the spatial levitation image 3 can be defined as a depth-shifting operation. The assignment of depth-shifting operations can also be arbitrarily set by the user. The spatial levitation image display device, for example, displays on the user-defined screen as follows: Figure 19G The information shown allows the user to select an operation and set it as a depth movement operation.

[0472] Figure 19H This indicates that an example of a depth movement operation has been set. Figure 19G The process for handling long-press operations. Figure 19H In the middle, as an example, to illustrate from Figure 19E Starting from state (A), the image 2001E of the character of the front spatial floating image 3E is moved to the rear spatial floating image 3D by the depth movement operation.

[0473] Space-suspended image display device Figure 3 The image control unit 1160 performs as follows Figure 19H The processing is as shown. The spatial levitation image display device detects touch events on the two layers of spatial levitation images 3 based on the air operation detection sensor 1351 and the air operation detection unit 1350. Here, the touch event "Down" refers to the user's finger touching the light plane formed by the air operation detection sensor 1351, which illuminates the plane of the spatial levitation image 3. In other words, the sensor detects a change from a state where the finger is not touching the plane of the spatial levitation image 3 to a state where it is touched (i.e., positioned in the air). Conversely, the touch event "Up" refers to the user's finger leaving the light plane formed by the sensor. In other words, the sensor detects a change from a state where the finger touched the plane of the spatial levitation image 3 to a state where it is not touched.

[0474] In step S19H1, the space levitation image display device detects a touch event "Down" on the front space levitation image 3E based on the air operation detection sensor 1351, etc. In this example, the detection target is the entire plane of the space levitation image 3E, any part thereof. When "Down" is detected, in step S19H2, the space levitation image display device starts counting a timer.

[0475] In step S19H3, the space levitation image display device detects a touch event Up on the front space levitation image 3E based on the air operation detection sensor 1351, etc. Upon detecting Up, in step S19H4, the space levitation image display device stops counting its timer. The time value counted by the timer is thus obtained. This time value is the time from touch Down to touch Up.

[0476] In step S19H5, the spatial levitation image display device determines whether the timer value is below a set threshold. If it is below the threshold ("Yes"), proceed to step S19H6; if it exceeds the threshold ("No"), proceed to step S19H8. In this example, the threshold for the specified time is 1 second.

[0477] If the process proceeds to step S19H6, the spatial levitation image display device determines that it is a touch operation within the same screen, and in step S19H7, the timer value is reset.

[0478] When the process proceeds to step S19H8, the spatial levitation image display device determines that it is a touch operation (i.e., a long press operation) corresponding to the depth movement operation, and in step S19H9, the timer value is reset.

[0479] Figure 19I Indicates about Figure 19H The display control example for a deep movement operation, as shown in step S19H6, is determined to be an operation within the same screen. The horizontal axis of the graph represents time, and the vertical axis represents the detection of touch events by sensors, etc. Initially, the sensor detection is in an OFF state. After detecting a touch event Down for 0 seconds, the sensor detection becomes ON. The graph shows the case where a touch event Up is detected 1 second before the threshold is set, and the sensor becomes OFF again. When such a touch detection ON state (in other words, continuously detecting Down) continues for less than the threshold, it is determined to be an operation within the same screen.

[0480] In this case, the Up time point 1 second prior is determined to be an operation within the same frame. Therefore, in the front-side spatial floating image 3E, the display of the character's image 2001E changes accordingly based on the touch operation (i.e., the operation within the same frame). For example, initially, the character's image 2001E displays an image of a basic pose. After the operation within the same frame is performed, the character's image 2001E displays an animated image of performing an action corresponding to the operation within the same frame (e.g., waving). On the other hand, because this operation is not a depth movement operation, the image of the rear-side spatial floating image 3D does not change before and after the Up time point; for example, it remains in a state where the character's image 2001D is not displayed.

[0481] Figure 19JIndicates about Figure 19H The display control example for a deep movement operation, as shown in step S19H8, is as follows: The horizontal axis of the figure represents time, and the vertical axis represents the detection of touch events by sensors, etc. Initially, the sensor detection is in the OFF state. After detecting a touch event Down for 0 seconds, the sensor detection becomes in the ON state. The figure shows the case where a touch event Up is detected after 1 second (threshold), and the sensor becomes OFF again. When such a touch detection ON state (in other words, continuously detecting Down) continues for more than the threshold, it is determined to be a deep movement operation corresponding to a long press operation.

[0482] In this case, the Up time point after 1 second is determined to be a depth movement operation. Therefore, the display of the character's images 2001E and 2001D in the front spatial floating image 3E and the rear spatial floating image 3D changes accordingly according to this touch operation (i.e., the depth movement operation). For example, the display of the character's image 2001E in the front spatial floating image 3E is set to OFF (not displayed), and the display of the corresponding character's image 2001D in the rear spatial floating image 3D is set to ON (displayed). This achieves the visual effect of the character moving from a spatial position in the front spatial floating image 3E to a spatial position in the rear spatial floating image 3D. In addition, during this movement, display control for the character's movement animation can also be added.

[0483] Examples of the depth movement operations and display controls for the aforementioned characters, such as... Figure 19A Similarly, the display controls for the button icons shown can also be adjusted simultaneously in terms of size, brightness, etc. For example, relative to the image 2001E of the character in front, the image 2001D of the character behind can be displayed slightly smaller, and its brightness can be displayed slightly lower. This further enhances the sense of distance.

[0484] <Example 5>

[0485] As Embodiment 5 of the present invention, a structural example of a spatial levitation image display device will be described. Embodiment 5 is similar to Embodiment 1 described above. Figures 1-4M ) and Example 2 ( Figure 14 Similarly, it is a spatial levitation image display device capable of displaying a single-layer spatial levitation image that floats from the optical system. In this embodiment, the differences from Embodiments 1 and 2 will be mainly described.

[0486] Figure 20A This illustrates the structure of the spatial levitation image display device in Embodiment 4. Figure 20A The structure is, for example, based on Figure 4F , Figure 14 and Figure 17CBased on the structure, it forms a single-layer spatial levitation image 3. This example structure is a vertical type, which is the type in which the user views the single-layer spatial levitation image 3 in the horizontal direction (y direction, arrow A direction), but it is of course not limited to this, and various variations can be adopted, such as a horizontal type, a type for viewing in the vertical direction or an oblique direction, etc.

[0487] Figure 20A The spatial levitation image display device is housed within a housing 1190. The aforementioned display device 1, polarization separation component 101B, and retroreflector 2 with a λ / 4 waveplate 21 are disposed at the rear portion 1190B in the y-direction. Image light (e.g., S-polarized light) reflected by the polarization separation component 101B is emitted forward in the y-direction. This image light is emitted, for example, via a transparent component 100 to the front portion 1190A of the housing 1190. Alternatively, it can be configured without the transparent component 100. The front portion 1190A has an upper portion 1190C and a lower portion 1190D, with a space exposed to the outside between the upper portion 1190C and the lower portion 1190D. The image light emitted to the front portion 1190A forms a real spatial levitation image 3 at a predetermined position, levied by a predetermined amount. This spatial levitation image 3 is configured in the xz plane.

[0488] An aerial operation detection sensor 1351 for sensing the levitation image 3 and a camera unit 1180 for sensing the spatial region including the levitation image 3 (particularly the upper surface of the stage 2010) are provided at the upper part 1190C. In this example, the aerial operation detection sensor 1351 is, for example, a sheet-like sensor, located at a position in the y-direction corresponding to the position of the levitation image 3 in the y-direction. The aerial operation detection sensor 1351 emits light downward in the vertical direction (z-direction) and receives reflected light from any object. The light generated by the aerial operation detection sensor 1351 covers the xz plane of the levitation image 3. Using the aerial operation detection sensor 1351 and the above-mentioned ( Figure 3 The aerial operation detection unit 1350, etc., can detect touch operations / aerial operations performed on the plane of the suspended image 3, as well as the presence and position of objects. That is, when an object is placed in contact with the display area of ​​the suspended image 3, the aerial operation detection sensor 1351 can detect the placement of the object.

[0489] Furthermore, the camera unit 1180, for example, is configured to capture a defined range downwards in the z-direction. The defined range of this camera covers the xz plane of the spatial levitation image 3 and the upper surface of the stage 2010 of the lower part 1190D, i.e., the xy plane.

[0490] A stage 2010 and a non-contact sensor 2020 are installed in the lower part 1190D. The stage 2010 has an xy plane. The spatial levitation image 3 is formed at a predetermined position on the upper side of the stage 2010. In addition, the user 230 can place an object on the xy plane of the stage 2010 (described later).

[0491] By using the aerial operation detection sensor 1351, it is possible to detect whether an object is placed on the stage 2010 and to determine its position, height, shape, etc. This function is not limited to the aerial operation detection sensor 1351; other types of sensors can also be used.

[0492] Furthermore, by using the camera unit 1180, functions such as detecting whether an object is placed on the stage 2010 and determining its position can be achieved. The camera unit 1180 is not limited to this; it can also capture images of the user 230's face, as described above.

[0493] The non-contact sensor 2020 is a sensor capable of detecting objects placed on the stage 2010. Specifically, it is a sensor capable of identifying objects placed on the stage 2010, such as RF tags. The non-contact sensor 2020 can be, for example, an RFID reader in hardware. It can also be implemented using NFC or similar technologies. When an object with an RFID tag is physically placed on the stage 2010, the non-contact sensor 2020, i.e., the RFID reader, can read the information stored in the RFID tag to identify the object.

[0494] Figure 20B This represents a planar view in the y-direction as seen from user 230. Figure 20A This is a schematic diagram of a structural example in which an aerial operation detection sensor 1351 (in other words, a planar sensor) senses the plane of the spatial levitation image 3 on the stage 2010, i.e., on the xz plane. Figure 20B In this state, it is assumed that no object (in other words, no physical object) is placed on the stage 2010. The air-operation detection sensor 1351 has, for example, multiple light-emitting units (not shown) and multiple light-receiving units arranged in the x-direction. Each light-emitting unit emits light a1 downwards in the z-direction as indicated by the arrow, for example, infrared light (non-display light). The emitted light a1 reaches the upper surface 2010a of the stage 2010 in the absence of any light-blocking objects, including the user's finger, is reflected, and returns upwards in the z-direction. The light-receiving unit receives this reflected light.

[0495] The airborne operation detection sensor 1351 can, for example, determine distance using a TOF (Time-of-Flight) method based on the time it takes for light a1 to travel from emission to return. For instance, distance d1 can be measured based on light a1 emitted from a light-emitting part at a certain location. That is, based on this distance d1, it can be determined that there is no light-blocking object on the stage 2010 at that position in the x-direction. On the other hand, suppose an object 2011 is placed, for example, in the area shown by the dashed line. In this case, light a2 emitted from a light-emitting part at a certain location is reflected by the upper surface of the object 2011, and distance d2 can be measured. That is, based on this distance d2, it can be determined that there is a light-blocking object 2011 on the stage 2010 at that position in the x-direction. Furthermore, the height h of the object 2011 (h = d1 - d2) can be calculated based on this distance d2.

[0496] in addition, Figure 20B In the example, there is a gap between the top edge of the spatial levitation image 3 and the bottom edge of the aerial operation detection sensor 1351, but the distance of this gap can also be made almost zero.

[0497] The sensing of the plane of the space-suspended image 3 by the aerial operation detection sensor 1351 is not limited to the examples described above. In other examples, the aerial operation detection sensor 1351 can be configured with the left and right sides of the space-suspended image 3 at a position in the lateral (x-direction), and the sensor emits light in the lateral (x-direction) direction for sensing. This structure can also detect the presence and approximate height of an object placed on the stage 2010.

[0498] Figure 20C This illustrates a display example of the spatially suspended image 3 on the aforementioned platform 2010. This display example is an instance of displaying the image 2041 of a character using animation. Figure 20C The image schematically illustrates a basic animation example in which the character's image 2041 moves on the platform 2010 while the platform 2010 is empty, and the image 3 is suspended in the first-level space. Figure 20C The illustration of the air operation detection sensor 1351 is omitted; only the light used for sensing is indicated by an arrow.

[0499] This example illustrates the movement of character 2041 to the left (in the -x direction). As part of the animation, on the xz plane of the suspended spatial image 3, the character's image 2041, for example, starts from image 2041a on the right side, moves to the left in the x direction to become image 2041b in the center position, and then moves further to the left to become image 2041c on the left side. Furthermore, the leftward arrow in the diagram is not part of the suspended spatial image 3 itself, but is simply labeled to illustrate the movement of character 2041; the same applies to the subsequent accompanying figures.

[0500] These images 2041, for example, can be derived from... Figure 3 The image is generated by the image control unit 1160. Alternatively, these images can be generated by the image control unit 1160 in cooperation with the control unit 1110 and the memory 1109. Animated images of character movements, etc., can be generated by rendering the virtual 3D space in which the character object is located. This also applies to image generation described below.

[0501] Figure 20D This represents a sensor detection example when an object (physical object) is placed on the stage 2010. Figure 20D In the example, a miniature model object mimicking a staircase is placed as object 2071 on platform 2010. For example, user 230 places object 2071 on platform 2010.

[0502] For the object 2071 placed on the stage 2010, the aerial operation detection sensor 1351 and other sensors detect and measure the height of each position in the x-direction on the xz plane of the space levitation image 3.

[0503] Here, there are cases where the type of object 2071 placed on the stage 2010 is not recognized, and cases where its type is recognized. In the case where it is not recognized, the spatial levitation image display device only uses the aerial operation detection sensor 1351 to determine the position, height, etc. of the object 2071. For example, it obtains information such as the height at position x1 is h1 and the height at position x2 is h2.

[0504] In contrast, when identifying the type of object 2071 place...

Claims

1. An aerial levitation image display device, characterized in that, include: The display unit displays images; An optical system that generates an aerial levitation image based on the image displayed on the display unit; and Sensors are used to detect aerial operations performed by the user on the aerial levitation image, wherein... The aerial levitation image is formed in two layers in the depth direction when viewed from the user's viewpoint. The front layer is referred to as the first aerial levitation image, and the rear layer is referred to as the second aerial levitation image. Display the image of the first object in the first aerial levitation image. Detect the aerial operations performed by the user on the first object image of the first aerial levitation image. If the aerial operation is detected, control is performed to change the display content of the first object image in the first aerial levitation image, and a second object image corresponding to the first object image is displayed in the second aerial levitation image.

2. The aerial levitation image display device as described in claim 1, characterized in that: The first state of the button is displayed in the first aerial levitation image as the first object image. The second state of the button is displayed as the second object image in the second aerial levitation image.

3. The aerial levitation image display device as described in claim 1, characterized in that: If the aerial operation is detected, control is performed to change the first object image in the first aerial levitation image in a gradually disappearing manner, and to make the second object image gradually appear in the second aerial levitation image.

4. The aerial levitation image display device as described in claim 1, characterized in that: The second object image of the second aerial levitation image is displayed with a size smaller than the first object image of the first aerial levitation image.

5. The aerial levitation image display device as described in claim 1, characterized in that: The brightness, luminance, or chroma of the second object image in the second aerial levitation image is made to be less than the brightness, luminance, or chroma of the first object image in the first aerial levitation image for display.

6. The aerial levitation image display device as described in claim 1, characterized in that: In the case of detecting a specified touch operation on the plane of the first aerial levitation image as part of the aerial operation performed on the first aerial levitation image, the display is controlled to move the first object image of the first aerial levitation image to the second object image of the second aerial levitation image.

7. The aerial levitation image display device as described in claim 6, characterized in that: The first state of displaying the character's image in the first aerial levitation image is used as the first object image. The second state of the character image is displayed as the second object image in the second aerial levitation image.

8. The aerial levitation image display device as described in claim 1, characterized in that: In the display unit and the optical system, The display screen of the display unit has a first image display area and a second image display area. include: First polarization separation component; First λ / 4 waveplate; First regression reflector; Second polarization separation component; Second λ / 4 waveplate; Mirror reflector; The third λ / 4 waveplate; and The second regression reflector, in which... Image light of a predetermined polarization emitted from the first image display area passes through the first polarization separating component. The transmitted image light then passes through the first λ / 4 waveplate and is reflected back onto the first retroreflector plate. The reflected image light then passes through the first λ / 4 waveplate, thus becoming image light of another polarization with a phase difference of 90° from the predetermined polarization. This other polarization image light is reflected by the first polarization separating component, and the reflected image light forms a real image, the second suspended image in the air, at a second position in the air. Image light of a predetermined polarization emitted from the second image display area passes through the second polarization separating component. The transmitted image light then passes through the second λ / 4 waveplate and is reflected on the mirror reflector. The reflected image light then passes through the second λ / 4 waveplate, thus becoming image light of another polarization with a phase difference of 90° from the predetermined polarization. This other polarization image light is reflected on the second polarization separating component. The reflected image light then passes through the third λ / 4 waveplate and is reflected back on the second retroreflector. The reflected image light then passes through the third λ / 4 waveplate, thus becoming image light of the predetermined polarization with a phase difference of 90° from the other polarization. This image light then passes through the second polarization separating component, and the transmitted image light then passes through the first polarization separating component. The transmitted image light forms a real image of the first suspended image in the air at a first position.

9. An aerial levitation image display device, characterized in that, include: The display unit displays images; An optical system that generates an aerial levitation image based on the image displayed on the display unit; and A sensor that detects the placement of objects at positions in contact with the display area of ​​the suspended aerial image, wherein, Detect the position and height of the object. Based on the detected position and height of the object, control is performed to change at least the position of the object image displayed in the aerial levitation image.

10. The aerial levitation image display device as described in claim 9, characterized in that: The character image is displayed as the object image in the suspended aerial image. Based on the detected position and height of the object, the character image is controlled to move away from the object or to move along the vertical upper side of the object.

11. The aerial levitation image display device as described in claim 9, characterized in that: Identify the types of objects configured in the aerial levitation image. The image of the character is displayed as the object image in the suspended aerial image. Based on the type of the identified object, control is performed to change the display content of the character image shown in the aerial levitation image.

12. The aerial levitation image display device as described in claim 11, characterized in that: Based on the identified type of object, control is performed to change the movement of the character image displayed in the aerial levitation image relative to the object.

13. The aerial levitation image display device as described in claim 11, characterized in that: Based on the type of the identified object, control is applied to change the clothing of the character image displayed in the aerial levitation image.

14. The aerial levitation image display device as described in claim 11, characterized in that: Based on the type of the identified object, control is performed to change the character itself displayed in the aerial levitation image.

15. The aerial levitation image display device as described in claim 9, characterized in that: A platform is provided below the location where the aerial image is formed, allowing the user to place the object on the platform.

16. The aerial levitation image display device as described in claim 11, characterized in that: It includes non-contact sensors that are capable of identifying the types of objects configured for the aerial levitation image.

17. The aerial levitation image display device as described in claim 9, characterized in that: The display unit and the optical system include: Polarization separation component; λ / 4 waveplate; and The return reflector, among which, Image light of a predetermined polarization emitted from the display unit passes through the polarization separating member, and then passes through the λ / 4 waveplate and is reflected back onto the retroreflector. The reflected image light then passes through the λ / 4 waveplate, thus becoming image light of another polarization with a phase difference of 90° from the original polarization. The image light of the other polarization is reflected on the polarization separating member, and the reflected image light forms a real image of the suspended image in the air at a predetermined position.

18. The aerial levitation image display device as described in claim 9, characterized in that: The aerial levitation image is formed in two layers in the depth direction when viewed from the user's viewpoint. The front layer is referred to as the first aerial levitation image, and the rear layer is referred to as the second aerial levitation image. Display the object image in the first aerial levitation image. Display the background image in the second aerial levitation image. Identify the types of objects configured in the aerial levitation image. Based on the type of the identified object, control is performed to change the background image displayed in the second aerial levitation image.

19. The aerial levitation image display device as described in claim 9, characterized in that: Including a second display device, In the depth direction when viewing the levitation image from the user's viewpoint, the display image formed by the screen of the second display device is superimposed on the levitation image at the depth side. The object image is displayed in the aerial levitation image. Display a background image on the screen of the second display device. Identify the types of objects configured in the aerial levitation image. Based on the type of the identified object, control is performed to change the background image displayed on the screen of the second display device.

20. The aerial levitation image display device as described in claim 18, characterized in that: In the display unit and the optical system, The display screen of the display unit has a first image display area and a second image display area. include: First polarization separation component; First λ / 4 waveplate; First regression reflector; Second polarization separation component; Second λ / 4 waveplate; Mirror reflector; The third λ / 4 waveplate; and The second regression reflector, in which... Image light of a predetermined polarization emitted from the first image display area passes through the first polarization separating component. The transmitted image light then passes through the first λ / 4 waveplate and is reflected back onto the first retroreflector plate. The reflected image light then passes through the first λ / 4 waveplate, thus becoming image light of another polarization with a phase difference of 90° from the predetermined polarization. This other polarization image light is reflected by the first polarization separating component, and the reflected image light forms a real image, the second suspended image in the air, at a second position in the air. Image light of a predetermined polarization emitted from the second image display area passes through the second polarization separating component. The transmitted image light then passes through the second λ / 4 waveplate and is reflected on the mirror reflector. The reflected image light then passes through the second λ / 4 waveplate, thus becoming image light of another polarization with a phase difference of 90° from the predetermined polarization. This other polarization image light is reflected on the second polarization separating component. The reflected image light then passes through the third λ / 4 waveplate and is reflected back on the second retroreflector. The reflected image light then passes through the third λ / 4 waveplate, thus becoming image light of the predetermined polarization with a phase difference of 90° from the other polarization. This image light then passes through the second polarization separating component, and the transmitted image light then passes through the first polarization separating component. The transmitted image light forms a real image of the first suspended image in the air at a first position.

Images