Aerial suspension image display device, portable terminal and display method

Abstract

The invention provides a better aerial suspension image display device. According to the invention, it is possible to contribute to "3 good health and welfare", "9 industry, innovation and infrastructure", "11 sustainable cities and communities" of sustainable development objectives (SDGs). The air-floating image display device performs control on the basis of a communication connection with a portable terminal of a user, for example, displays a display image displayed on a screen of the portable terminal as a display image in the air-floating image, and performs control on the basis of detection and determination of an operation performed by the user on the display image displayed on the screen of the portable terminal. A predetermined process associated with a display image on a screen is executed, and the process is also reflected in a display image of an aerial floating image. In addition, the present system is capable of setting and controlling which device operation and detection become valid / invalid.

Description

Technical Field

[0001] This invention relates to an aerial levitation image display device. Background Technology

[0002] Regarding the technology of displaying information suspended in the air, for example, it is disclosed in Patent Document 1.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2019-128722 Summary of the Invention

[0006] The technical problem that the invention aims to solve

[0007] However, the content disclosed in Patent Document 1 does not fully consider how to obtain practical brightness and quality of aerial levitation images, or how to make users enjoy watching aerial levitation images more pleasantly.

[0008] The purpose of this invention is to provide a better aerial levitation image display device.

[0009] Technical means to solve the problem

[0010] To solve the above problems, for example, the structure described in the claimed technical solution can be adopted. This application includes various means to solve the above problems, one example of which can be configured as follows. One embodiment is an aerial levitation image display device, including: an image processing unit; a display unit that displays an image processed by the image processing unit; an optical system that generates an aerial levitation image based on the image displayed by the display unit; a detection unit that detects contact between an object and the display area of ​​the aerial levitation image; and a communication unit that communicates with a user's portable terminal, and displays a target image specified on one side of the portable terminal as an aerial levitation image when contact between the portable terminal and the display area is detected.

[0011] Invention Effects

[0012] The present invention enables the realization of a better aerial levitation image display device. Other technical problems, technical features, and technical effects will become clear in the following description of the embodiments. Attached Figure Description

[0013] Figure 1 This figure shows an example of the usage mode of a spatial levitation image display device according to an embodiment of the present invention.

[0014] Figure 2A This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0015] Figure 2B This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0016] Figure 2C This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0017] Figure 2D This is a diagram illustrating an example of the main structure and retroreflective part structure of an aerial levitation image display device according to an embodiment of the present invention.

[0018] Figure 2E This is a projection diagram of the retroreflector that constitutes the aerial suspended image display device in one embodiment of the present invention.

[0019] Figure 2F This is a top view of the retroreflector that constitutes the aerial suspended image display device in one embodiment of the present invention.

[0020] Figure 2G This is a perspective view showing the corner reflector included in the retroreflector panel constituting the aerial suspended image display device in one embodiment of the present invention.

[0021] Figure 2H This is a top view showing the corner reflector included in the retroreflector that constitutes the aerial levitation image display device in one embodiment of the present invention.

[0022] Figure 2I This is a side view showing the corner reflector included in the retroreflector that constitutes the aerial levitation image display device in one embodiment of the present invention.

[0023] Figure 3 This is a diagram illustrating a structural example of a spatial levitation image display device according to an embodiment of the present invention.

[0024] Figure 4A This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0025] Figure 4B This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0026] Figure 4C This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0027] Figure 4D This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0028] Figure 4E This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0029] Figure 4F This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0030] Figure 4G This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0031] Figure 4H This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0032] Figure 4I This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0033] Figure 4J This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0034] Figure 4K This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0035] Figure 4L This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0036] Figure 4M This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0037] Figure 4N This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0038] Figure 4O This is a diagram illustrating an example of the structure of a spatial levitation image display device according to an embodiment of the present invention.

[0039] Figure 5 This is a cross-sectional view illustrating an example of the specific structure of a light source device according to an embodiment of the present invention.

[0040] Figure 6 This is a cross-sectional view illustrating an example of the specific structure of a light source device according to an embodiment of the present invention.

[0041] Figure 7 This is a cross-sectional view illustrating an example of the specific structure of a light source device according to an embodiment of the present invention.

[0042] Figure 8 This is a configuration diagram showing the main parts of a spatial levitation image display device according to an embodiment of the present invention.

[0043] Figure 9 This is a cross-sectional view showing the structure of a display device according to an embodiment of the present invention.

[0044] Figure 10 This is a cross-sectional view showing the structure of a display device according to an embodiment of the present invention.

[0045] Figure 11 This is an explanatory diagram illustrating the light source diffusion characteristics of an image display device according to an embodiment of the present invention.

[0046] Figure 12 This is an explanatory diagram illustrating the diffusion characteristics of an image display device according to an embodiment of the present invention.

[0047] Figure 13A This is an illustrative diagram illustrating an example of the technical problem to be solved by image processing according to an embodiment of the present invention.

[0048] Figure 13B This is an illustrative diagram illustrating an example of image processing according to an embodiment of the present invention.

[0049] Figure 13C This is an illustrative diagram illustrating an example of image display processing according to an embodiment of the present invention.

[0050] Figure 13D This is an illustrative diagram illustrating an example of image display processing according to an embodiment of the present invention.

[0051] Figure 14 This is a diagram illustrating an example of the main structure and retroreflective part structure of a spatial levitation image display device according to an embodiment of the present invention.

[0052] Figure 15 This is a diagram illustrating the structure of a spatial levitation image display system according to an embodiment of the present invention.

[0053] Figure 16 This is a diagram illustrating a schematic representation of a spatial levitation image display system according to an embodiment of the present invention.

[0054] Figure 17 This is a diagram illustrating the structure of a portable terminal according to an embodiment of the present invention.

[0055] Figure 18A This is a perspective view illustrating the insertion / contact operation of a portable terminal on a spatially suspended image in one embodiment of the present invention.

[0056] Figure 18BThis is an XY plane diagram illustrating the insertion / contact operation of a portable terminal on a spatially suspended image in one embodiment of the present invention.

[0057] Figure 18C This is a YZ plane diagram illustrating the insertion / contact operation of a portable terminal on a spatially suspended image in one embodiment of the present invention.

[0058] Figure 18D This is an XZ-plan view illustrating the insertion / contact operation of a portable terminal on a spatially suspended image in one embodiment of the present invention.

[0059] Figure 19A This is a perspective view showing the posture, etc., of a portable terminal during insertion / contact operation of a spatially suspended image in one embodiment of the present invention.

[0060] Figure 19B This is an XY plan view illustrating an example of the posture, etc., of a portable terminal during insertion / contact operation of a spatially suspended image in one embodiment of the present invention.

[0061] Figure 19C This is a YZ plan view illustrating an example of the posture, etc., of a portable terminal during insertion / contact operation of a spatially suspended image in one embodiment of the present invention.

[0062] Figure 19D This is an XZ plan view illustrating an example of the posture, etc., of a portable terminal during insertion / contact operation of a spatially suspended image in one embodiment of the present invention.

[0063] Figure 19E This is a YZ plan view illustrating an example of the posture, etc., of a portable terminal during insertion / contact operation of a spatially suspended image in one embodiment of the present invention.

[0064] Figure 19F This is a YZ plan view illustrating an example of the posture, etc., of a portable terminal during insertion / contact operation of a spatially suspended image in one embodiment of the present invention.

[0065] Figure 20A This is a YZ plan view showing an example of the position, etc., of a portable terminal during an insertion / contact operation of a spatially suspended image in one embodiment of the present invention.

[0066] Figure 20B This is an XY plan view illustrating an example of the position, etc., of a portable terminal during an insertion / contact operation of a spatially suspended image in one embodiment of the present invention.

[0067] Figure 20C This is a YZ plan view showing an example of the position, etc., of a portable terminal during an insertion / contact operation of a spatially suspended image in one embodiment of the present invention.

[0068] Figure 21AThis is an explanatory diagram illustrating an example of displaying a guide image on a spatially suspended image according to an embodiment of the present invention.

[0069] Figure 21B This is an explanatory diagram illustrating an example of displaying a guide image on a spatially suspended image according to an embodiment of the present invention.

[0070] Figure 22A This is an XY plan view illustrating an example of a portable terminal contacting a spatially suspended image in one embodiment of the present invention.

[0071] Figure 22B This is an XZ plan view showing an example of an invisible area generated due to contact between a portable terminal and a spatially suspended image in one embodiment of the present invention.

[0072] Figure 22C This is a YZ plan view illustrating an example of light blocking caused by contact between a portable terminal and a spatially suspended image in one embodiment of the present invention.

[0073] Figure 23A This is an xy-plane diagram illustrating an example of displaying a guide image on a spatially suspended image according to an embodiment of the present invention, and an example of an invisible area generated due to contact with a portable terminal.

[0074] Figure 23B This is an xy-plane diagram illustrating an example of displaying a guide image on a spatially suspended image and an example of limiting the display area, according to an embodiment of the present invention.

[0075] Figure 24A This is an xy-plane view illustrating an example of displaying a guide image on a spatially suspended image according to an embodiment of the present invention.

[0076] Figure 24B This is an xy-plane view illustrating an example of displaying a guide image on a spatially suspended image according to an embodiment of the present invention.

[0077] Figure 24C This is an explanatory diagram illustrating an example of displaying a guide image on a spatially suspended image and an example of limiting the display area, according to an embodiment of the present invention.

[0078] Figure 24D This is an explanatory diagram illustrating an example of a guide sound output according to an embodiment of the present invention.

[0079] Figure 25A This is a YZ plane cross-sectional view showing an example of the sensor configuration in a spatial levitation image display device according to an embodiment of the present invention.

[0080] Figure 25B This is an xy-plane view illustrating an example of the structure of an airborne operation detection sensor according to an embodiment of the present invention.

[0081] Figure 25C This is a YZ plane cross-sectional view showing an example of the sensor configuration in a spatial levitation image display device according to an embodiment of the present invention.

[0082] Figure 26A This is an explanatory diagram showing an example of an application screen display in a portable terminal according to an embodiment of the present invention.

[0083] Figure 26B This is an explanatory diagram showing an example of an application screen display in a portable terminal according to an embodiment of the present invention.

[0084] Figure 26C This is an explanatory diagram showing an example of an application screen display in a portable terminal according to an embodiment of the present invention.

[0085] Figure 27A This is an explanatory diagram illustrating an example of QR code display in a portable terminal according to an embodiment of the present invention.

[0086] Figure 27B This is a YZ plane cross-sectional view illustrating an example of a portable terminal capturing a QR code in a spatial levitation image display device according to an embodiment of the present invention.

[0087] Figure 28A This is a perspective view illustrating an example of QR code display in a spatially suspended image according to an embodiment of the present invention.

[0088] Figure 28B This is a YZ plane cross-sectional view showing an example of a QR code for a portable terminal capturing and displaying spatial levitation images according to an embodiment of the present invention.

[0089] Figure 28C This is a perspective view illustrating an example of QR code display on the screen of a second display device according to an embodiment of the present invention.

[0090] Figure 29 This is a diagram illustrating a structural example of a system formed by a portable terminal and a spatially suspended image display device in one embodiment of the present invention.

[0091] Figure 30 This figure illustrates an example of a dual display of a portable terminal and a spatial levitation image display device according to an embodiment of the present invention.

[0092] Figure 31 This is a diagram illustrating the basic control flow of a system according to an embodiment of the present invention.

[0093] Figure 32A This is a diagram illustrating an example of a system connection display / settings according to an embodiment of the present invention.

[0094] Figure 32B This is a diagram illustrating an example (mode) of the display / settings after system connection according to an embodiment of the present invention.

[0095] Figure 33 This is a diagram illustrating a mode setting GUI example (1) of a screen display example for a portable terminal in one embodiment of the present invention.

[0096] Figure 34 This is a diagram illustrating a mode setting GUI example (2) of a screen display example for a portable terminal in one embodiment of the present invention.

[0097] Figure 35 This is a diagram illustrating an example of the functional module structure of a system according to an embodiment of the present invention.

[0098] Figure 36 This diagram illustrates an example of the screen structure of a spatial levitation image display device and an example of the screen structure of a portable terminal, according to an embodiment of the present invention.

[0099] Figure 37 This is a diagram of Example 1 illustrating a technical problem and solution of an embodiment of the present invention.

[0100] Figure 38 This is a figure illustrating Example 2, which shows a technical problem and solution of an embodiment of the present invention.

[0101] Figure 39 This is Figure 3, illustrating the technical problem and solution of an embodiment of the present invention.

[0102] Figure 40 This is Figure 4, illustrating the technical problem and solution of an embodiment of the present invention.

[0103] Figure 41 This is a diagram illustrating an example of image display of a spatial levitation image display device and a portable terminal according to an embodiment of the present invention.

[0104] Figure 42 This diagram illustrates an example of a sensor used in the operation and control of a controlled object in one embodiment of the present invention.

[0105] Figure 43 This is a diagram of Example 1 illustrating the posture of a spatial levitation image display device according to an embodiment of the present invention.

[0106] Figure 44 This is Figure 2, illustrating the posture of a spatial levitation image display device according to an embodiment of the present invention.

[0107] Figure 45 This is Figure 3, which illustrates the posture of a spatial levitation image display device according to an embodiment of the present invention.

[0108] Figure 46 This is a diagram illustrating a timing example (1) of mode 1 of an embodiment of the present invention.

[0109] Figure 47 This is a diagram illustrating a timing example (2) of mode 1 of an embodiment of the present invention.

[0110] Figure 48 This is a diagram illustrating a timing example (3) of mode 1 of an embodiment of the present invention.

[0111] Figure 49 This is a diagram illustrating a timing example (4) of mode 1 of an embodiment of the present invention.

[0112] Figure 50 This is a diagram illustrating a timing example (1) of mode 2 of an embodiment of the present invention.

[0113] Figure 51 This is a diagram illustrating a timing example (2) of mode 2 of an embodiment of the present invention.

[0114] Figure 52 This is a diagram illustrating a timing example (1) of mode 3 of an embodiment of the present invention.

[0115] Figure 53 This is a diagram illustrating a timing example (2) of mode 3 of an embodiment of the present invention.

[0116] Figure 54 This is a diagram illustrating a timing example (1) of mode 5 of an embodiment of the present invention.

[0117] Figure 55 This is a diagram illustrating a timing example (1) of mode 7 of an embodiment of the present invention.

[0118] Figure 56 This is a diagram illustrating a timing example (2) of mode 7 of an embodiment of the present invention.

[0119] Figure 57 This is a diagram illustrating a timing example (3) of mode 7 of an embodiment of the present invention.

[0120] Figure 58 This is a diagram illustrating a timing example (4) of mode 7 of an embodiment of the present invention.

[0121] Figure 59 This is a diagram illustrating a timing example (1) of mode 8 of an embodiment of the present invention.

[0122] Figure 60 This is a diagram illustrating a timing example (2) of mode 8 of an embodiment of the present invention.

[0123] Figure 61This is a diagram illustrating a timing example (1) of mode 9 of an embodiment of the present invention.

[0124] Figure 62 This is a diagram illustrating a timing example (2) of mode 9 of an embodiment of the present invention.

[0125] Figure 63 This is a diagram illustrating a timing example (1) of mode 11 of an embodiment of the present invention. Detailed Implementation

[0126] The embodiments of the present invention are described in detail below based on the accompanying drawings. However, the present invention is not limited to the description of the embodiments, and those skilled in the art can make various changes and modifications within the scope of the technical concept disclosed in this specification. Furthermore, in all the drawings used to illustrate the present invention, parts having the same function are labeled with the same reference numerals, and sometimes repeated descriptions are omitted.

[0127] The following embodiments relate to an image display device that enables an image formed by image light from an image light source to be transmitted through a transparent component such as glass used to separate space, and displayed outside the transparent component as a spatially suspended image. Furthermore, in the following description of the embodiments, the term "spatially suspended image" is used to describe an image suspended in space. Alternatively, it can be expressed as "aerial image," "spatial image," "aerially suspended image," "spatially suspended optical image displaying an image," "aerially suspended optical image displaying an image," etc. The term "spatially suspended image" primarily used in the description of the embodiments is taken as a representative example of these terms.

[0128] According to the following embodiments, a good image display device can be implemented in, for example, bank ATMs, station ticket machines, digital signage, etc. For example, while touch panels are commonly used in bank ATMs and station ticket machines, transparent glass or light-transmitting panels can also be used to display high-resolution image information in a spatially suspended state. In this case, by reducing the divergence angle of the emitted image light to an acute angle, and then unifying it to a specific polarization, only the light normally reflected by the retroreflector can be efficiently reflected. Therefore, light utilization efficiency is high, and ghosting, which is a problem in existing retroreflection methods besides the main spatially suspended image, can be suppressed, resulting in a clear spatially suspended image. Furthermore, by using a device including the light source of this embodiment, a novel and highly usable spatially suspended image display device (spatially suspended image display system) that significantly reduces power consumption can be provided. Additionally, for example, a vehicle-use spatially suspended image display device capable of displaying so-called one-way spatially suspended images can be provided, which can be viewed from inside and / or outside the vehicle.

[0129] <Example 1>

[0130] <An example of the usage of a spatial levitation image display device>

[0131] Figure 1 This diagram illustrates an example of the usage mode of a spatial levitation image display device according to an embodiment of the present invention, and also shows the overall structure of the spatial levitation image display device of this embodiment. The specific structure of the spatial levitation image display device will be described using... Figures 2A to 2I In a detailed description, light with narrow-angle pointing characteristics and specific polarization emitted from the image display device 1 serves as an image beam. After reflection by the optical system within the spatial levitation image display device, it first enters the retroreflector plate 2. After retroreflection, it passes through the transparent component 100 (glass, etc.), forming a real aerial image (spatial levitation image 3) on the outer side of the glass surface. Furthermore, in the following embodiments, the retroreflector plate 2 (retroreflective reflector) is used as an example of a retroreflective component. However, the retroreflector plate 2 of the present invention is not limited to a planar plate; it is used as an example. The concept includes a sheet-like retroreflective component that can be attached to a planar or non-planar component, and an entire assembly obtained by attaching a sheet-like retroreflective component to a planar or non-planar component. Furthermore, the light reflected by the retroreflector plate 2 has optical characteristics capable of imaging, so the retroreflector plate 2 can also be expressed as an imaging optical component or an imaging optical plate.

[0132] Furthermore, in shops and similar establishments, display windows (also known as "window glass") 105, constructed from translucent components such as glass, divide the space. The spatial levitation image display device according to this embodiment can display levitation images unidirectionally to the exterior and / or interior of the shop (space) through these transparent components.

[0133] Figure 1 In this context, the inner side of the window glass 105 (inside the shop) is considered as the depth direction, and its outer side (e.g., the sidewalk) as the near side. On the other hand, light can also be reflected by providing a mechanism on the window glass 105 that performs specific polarization reflection, thus forming an aerial image at a desired location inside the shop.

[0134] <Example of the structure of the optical system of a spatial levitation image display device>

[0135] Figure 2A This diagram illustrates an example of the structure of the optical system of a spatial levitation image display device according to an embodiment of the present invention. Figure 2A To explain the structure of the spatial levitation image display device in more detail. For example... Figure 2A As shown in (1), a display device 1 is provided in the oblique direction of a transparent component 100 such as glass, which diffuses image light of a specific polarization in a narrow angle. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates light of a specific polarization with narrow angle diffusion characteristics.

[0136] Image light of a specific polarization from display device 1 is reflected by a polarization separating member 101 (in the figure, the polarization separating member 101 is formed as a sheet and attached to the transparent member 100) which has a film that selectively reflects the image light of the specific polarization and is disposed on the transparent member 100, and then incident on the retroreflector 2. A λ / 4 waveplate 21 is disposed on the image light incident surface of the retroreflector 2. The image light passes through the λ / 4 waveplate 21 twice, once when it is incident on the retroreflector 2 and once when it is emitted, thereby changing from a specific polarization to another polarization. Here, the polarization separating member 101, which selectively reflects the image light of the specific polarization, has the property of transmitting light of the other polarization after polarization change, so the image light of the specific polarization after polarization change passes through the polarization separating member 101. The image light after passing through the polarization separating member 101 forms a spatially suspended image 3 with a real image on the outside of the transparent member 100. In addition, regarding the main ray of the image light incident on the retroreflector 2, Figure 2A The example described is an incident angle of 90° relative to the retroreflector 2. However, the incident angle of the principal ray of the image light relative to the retroreflector 2 is not limited to 90°; for example, 90° ± 15° can also be used.

[0137] This section explains Figure 2A This is the first example of polarization design in an optical system. For instance, it can be configured such that S-polarized image light is emitted from a display device 1 to a polarization separation member 101, which has the characteristic of reflecting S-polarized light and transmitting P-polarized light. In this case, the S-polarized image light arriving at the polarization separation member 101 from the display device 1 is reflected by the polarization separation member 101 and goes to the retroreflector 2. When the image light is reflected on the retroreflector 2, it passes twice through the λ / 4 waveplate 21 disposed on the incident surface of the retroreflector 2, so the image light is converted from S-polarized light to P-polarized light. The image light converted to P-polarized light goes to the polarization separation member 101 again. Here, since the polarization separation member 101 has the characteristic of reflecting S-polarized light and transmitting P-polarized light, the P-polarized image light passes through the polarization separation member 101 and through the transparent member 100. The image light passing through the transparent component 100 is generated by the retroreflector 2. Therefore, at the position where the polarization separation component 101 and the displayed image of the display device 1 are in a mirror relationship, an optical image of the displayed image of the display device 1, namely the spatially suspended image 3, is formed. This polarization design enables the spatially suspended image 3 to be formed well.

[0138] Next, the explanation Figure 2AThis is a second example of polarization design in an optical system. For instance, it can be configured such that P-polarized image light is emitted from display device 1 to polarization separation member 101, which has the characteristic of reflecting P-polarized light and transmitting S-polarized light. In this case, the P-polarized image light arriving at polarization separation member 101 from display device 1 is reflected by polarization separation member 101 and goes to retroreflector 2. When the image light is reflected on retroreflector 2, it passes twice through λ / 4 waveplate 21 disposed on the incident surface of retroreflector 2, so the image light is converted from P-polarized light to S-polarized light. The image light converted to S-polarized light goes to polarization separation member 101 again. Here, since polarization separation member 101 has the characteristic of reflecting P-polarized light and transmitting S-polarized light, S-polarized image light passes through polarization separation member 101 and through transparent member 100. The image light passing through the transparent component 100 is generated by the retroreflector 2. Therefore, at the position where the polarization separation component 101 and the displayed image of the display device 1 are in a mirror relationship, an optical image of the displayed image of the display device 1, namely the spatially suspended image 3, is formed. This polarization design enables the spatially suspended image 3 to be formed well.

[0139] Furthermore, the light forming the spatial levitation image 3 is a collection of rays converged from the retroreflector 2 to the optical image of the spatial levitation image 3. These rays continue to travel in a straight line after passing through the optical image of the spatial levitation image 3. Therefore, the spatial levitation image 3 differs from the diffused image light formed on a screen using a conventional projector; it is a highly directional image. Thus, in Figure 2A In this structure, when a user views from the direction of arrow A, the spatially suspended image 3 can be seen as a bright image. However, when other people view from the direction of arrow B, the spatially suspended image 3 cannot be seen as an image at all. This feature is very suitable for systems that display images requiring high security, and systems that display highly confidential images that need to be kept secret from people facing the user.

[0140] Furthermore, depending on the performance of the retroreflector 2, the polarization axis of the reflected image light may not be uniform. Additionally, the reflection angle may also be inconsistent. Such inconsistent light may not maintain the intended polarization state and travel angle. For example, light deviating from the intended polarization state and travel angle may directly re-enter the image display surface of the liquid crystal display panel 11 from the position of the retroreflector 2 without passing through the polarization separation component. After being reflected by components within the spatial levitation image display device, this light deviating from the intended polarization state and travel angle may re-enter the image display surface of the liquid crystal display panel 11. This re-entry of the image display surface of the liquid crystal display panel 11 is reflected again on the image display surface constituting the display device 1, potentially causing ghosting and reducing the image quality of the spatial levitation image. Therefore, in this embodiment, an absorptive polarizer 12 can be provided on the image display surface of the display device 1. By allowing the image light emitted from the display device 1 to pass through the absorptive polarizer 12, and by using the absorptive polarizer 12 to absorb the reflected light returning from the polarization separation member 101, the aforementioned re-reflection can be suppressed. This prevents image quality degradation caused by ghosting of spatially suspended images. Specifically, if a structure is adopted in which S-polarized image light is emitted from the display device 1 to the polarization separation member 101, then the absorptive polarizer 12 can be a polarizer that absorbs P-polarized light. Alternatively, if a structure is adopted in which P-polarized image light is emitted from the display device 1 to the polarization separation member 101, then the absorptive polarizer 12 can be a polarizer that absorbs S-polarized light.

[0141] The polarization separation component 101 described above can be formed, for example, by a reflective polarizer or a multilayer metal film that reflects a specific polarization.

[0142] Next, in Figure 2A In (2), the retroreflector 2, as a representative example, illustrates an example of the surface shape of the retroreflector. It is composed of regularly arranged hexagonal prisms. Light incident inside is reflected on the walls and bottom of the hexagonal prisms to become retroreflected light, which is emitted in the direction corresponding to the incident light, and displays a spatial levitation image based on the image displayed on the display device 1.

[0143] The resolution of this spatial levitation image depends not only on the resolution of the liquid crystal display panel 11, but also significantly on... Figure 2A (2) shows the shape D and spacing P of the retroreflective portion of the retroreflective plate 2. For example, when using a 7-inch WUXGA (1920×1200 pixels) liquid crystal display panel, even if one pixel (one triplet) is about 80μm, if the diameter D of the retroreflective portion is 240μm and the spacing is 300μm, then one pixel of the spatial levitation image is equivalent to 300μm. Therefore, the effective resolution of the spatial levitation image is reduced to about 1 / 3.

[0144] Therefore, in order to make the resolution of the spatially suspended image the same as that of the display device 1, it is preferable to make the diameter and spacing of the retroreflective portion close to one pixel of the liquid crystal display panel. On the other hand, in order to suppress moiré patterns caused by the pixels of the retroreflective plate and the liquid crystal display panel, the spacing ratio of each can be designed to deviate from one pixel by an integer multiple. In addition, the shape can be configured such that neither side of the retroreflective portion coincides with either side of one pixel of the liquid crystal display panel.

[0145] Furthermore, the surface shape of the retroreflector in this embodiment is not limited to the examples described above. Various surface shapes can be used to achieve retroreflective properties. Specifically, a retroreflective element obtained by periodically arranging triangular pyramidal prisms, hexagonal pyramidal prisms, other polygonal prisms, or combinations thereof on the surface of the retroreflector in this embodiment can be provided. Alternatively, a retroreflective element formed by periodically arranging these prisms and creating cubic angles can also be provided on the surface of the retroreflector in this embodiment. This can also be expressed as a corner reflector array or a multifaceted reflector array. Alternatively, a capsule lens-type retroreflective element obtained by periodically arranging glass microspheres on the surface of the retroreflector in this embodiment can also be provided. Detailed structures of these retroreflective elements can be derived using existing technology, so detailed descriptions are omitted. Specifically, the technology disclosed in Japanese Patent Application Publication No. 2001-33609, Japanese Patent Application Publication No. 2001-264525, Japanese Patent Application Publication No. 2005-181555, Japanese Patent Application Publication No. 2008-70898, and Japanese Patent Application Publication No. 2009-229942 may be used.

[0146] <Example 1 of other structural features of the optical system of a spatial levitation image display device>

[0147] For other structural examples of the optical system of a spatial levitation image display device, using Figure 2B Please provide an explanation. Additionally... Figure 2B The text is marked with the same meaning. Figure 2A The same reference numerals have the same structure as Figure 2A Same function and structure. To simplify the explanation, repeated descriptions of such structures are omitted.

[0148] Figure 2B optical system and Figure 2A Similarly, image light with a specific polarization is output from display device 1. This image light with a specific polarization output from display device 1 is input to polarization separation component 101B. Polarization separation component 101B is a component that selectively transmits image light with a specific polarization. Polarization separation component 101B and... Figure 2AUnlike the transparent component 100, the polarization separating component 101B is not integrally formed with the transparent component 100 but is independently plate-shaped. Therefore, the polarization separating component 101B can also be described as a polarization separating plate. For example, the polarization separating component 101B can be configured as a reflective polarizer formed by attaching a polarization separating sheet to the transparent component. Alternatively, it can be formed on the transparent component using a multilayer metal film or the like that selectively transmits specific polarizations and reflects other specific polarizations. Figure 2B In this configuration, the polarization separation component 101B is configured to transmit image light of a specific polarization output from the display device 1.

[0149] Image light passing through polarization separation component 101B is incident on retroreflector 2. A λ / 4 waveplate 21 is provided on the image light incident surface of retroreflector. The image light passes through the λ / 4 waveplate 21 twice, once when it is incident on the retroreflector and once when it is emitted, thereby undergoing polarization transformation from a specific polarization to another polarization. Here, polarization separation component 101B has the property of reflecting the polarized light of the other polarization after polarization transformation by λ / 4 waveplate 21, so the polarization-transformed image light is reflected on polarization separation component 101B. The image light reflected on polarization separation component 101B passes through transparent component 100, forming a spatially suspended real image 3 on the outside of transparent component 100.

[0150] This section explains Figure 2B This is the first example of polarization design in an optical system. For instance, it can be configured such that P-polarized image light is emitted from display device 1 to polarization separation member 101B, which has the characteristic of reflecting S-polarized light and transmitting P-polarized light. In this case, the P-polarized image light arriving at polarization separation member 101B from display device 1 passes through polarization separation member 101B and goes to retroreflector 2. When the image light is reflected on retroreflector 2, it passes twice through λ / 4 waveplate 21 disposed on the incident surface of retroreflector 2, so the image light is converted from P-polarized light to S-polarized light. The image light converted to S-polarized light goes to polarization separation member 101B again. Here, since polarization separation member 101B has the characteristic of reflecting S-polarized light and transmitting P-polarized light, the S-polarized image light is reflected on polarization separation member 101B and passes through transparent member 100. The image light passing through the transparent component 100 is generated by the retroreflector 2. Therefore, at the position where the polarization separation component 101B and the displayed image of the display device 1 are in a mirror relationship, an optical image of the displayed image of the display device 1, namely the spatially suspended image 3, is formed. This polarization design enables the spatially suspended image 3 to be formed well.

[0151] Next, the explanation Figure 2BThis is a second example of polarization design in an optical system. For instance, it can be configured such that S-polarized image light is emitted from display device 1 to polarization separation member 101B, which has the characteristic of reflecting P-polarized light and transmitting S-polarized light. In this case, the S-polarized image light arriving at polarization separation member 101B from display device 1 passes through polarization separation member 101B and goes to retroreflector 2. When the image light is reflected on retroreflector 2, it passes twice through λ / 4 waveplate 21 disposed on the incident surface of retroreflector 2, so the image light is converted from S-polarized light to P-polarized light. The image light converted to P-polarized light goes to polarization separation member 101B again. Here, since polarization separation member 101B has the characteristic of reflecting P-polarized light and transmitting S-polarized light, the P-polarized image light is reflected on polarization separation member 101B and passes through transparent member 100. The image light passing through the transparent component 100 is generated by the retroreflector 2. Therefore, at the position where the polarization separation component 101B and the displayed image of the display device 1 are in a mirror relationship, an optical image of the displayed image of the display device 1, namely the spatially suspended image 3, is formed. This polarization design enables the spatially suspended image 3 to be formed well.

[0152] in addition, Figure 2B In this configuration, the image display surface of the display device 1 and the surface of the retroreflector 2 are arranged parallel to each other. The polarization separation member 101B is configured to be tilted at an angle α (e.g., 30°) relative to the image display surface of the display device 1 and the surface of the retroreflector 2. Thus, when reflected at the polarization separation member 101B, the direction of travel of the image light reflected at the polarization separation member 101B (the direction of the principal ray of the image light) differs from the direction of travel of the image light incident from the retroreflector 2 by an angle β (e.g., 60°). By employing this structure, in... Figure 2B In the optical system, image light is output to the outside of the transparent component 100 at a specified angle as shown in the figure, forming a spatially suspended image 3 of real image. Figure 2B In this structure, when a user views from the direction of arrow A, the spatially suspended image 3 can be seen as a bright image. However, when other people view from the direction of arrow B, the spatially suspended image 3 cannot be seen as an image at all. This feature is very suitable for systems that display images requiring high security, and systems that display highly confidential images that need to be kept secret from people facing the user.

[0153] As explained above, Figure 2B The optical system is with Figure 2A Optical systems with different structures, but capable of operating with... Figure 2A The optical system also forms a good spatial levitation image.

[0154] Additionally, an absorptive polarizer can be provided on the surface of the transparent component 100 on the side of the polarization separation member 101B. This absorptive polarizer can be one that allows the polarized transmission of image light from the polarization separation member 101B while absorbing polarizations that are 90° out of phase with the image light from the polarization separation member 101B. This allows for sufficient transmission of image light used to form the spatial levitation image 3, while reducing external light incident from the spatial levitation image 3 side of the transparent component 100 by approximately 50%. Consequently, the amount of light generated by external light incident from the spatial levitation image 3 side of the transparent component 100 can be reduced. Figure 2B Stray light within the optical system.

[0155] <Example 2 of other structural features of the optical system of a spatial levitation image display device>

[0156] For other structural examples of the optical system of a spatial levitation image display device, using Figure 2C Please provide an explanation. Additionally... Figure 2C The text is marked with the same meaning. Figure 2B The same reference numerals have the same structure as Figure 2B Same function and structure. To simplify the explanation, repeated descriptions of such structures are omitted.

[0157] Figure 2C Optical system relative to Figure 2B The only difference in the optical system is the arrangement angle of the polarization separation component 101B relative to the image display surface of the display device 1 and the surface of the retroreflector 2. All other structures are the same. Figure 2B Since the optical systems are the same, repeated descriptions are omitted. Figure 2C The polarization design of the optical system is also related to Figure 2B Since the polarization design of the optical system is the same, repeated explanations are omitted.

[0158] exist Figure 2C In the optical system, the polarization separation component 101B is configured to be tilted at an angle α relative to the image display surface of the display device 1 and the surface of the retroreflector 2. Figure 2CIn this configuration, the angle α is 45°. With this structure, when reflected from the polarization separating member 101B, the angle β between the direction of travel of the image light incident from the retroreflector 2 (the direction of the principal ray of the image light) and the direction of travel of the image light reflected from the polarization separating member 101B (the direction of the principal ray of the image light) is 90°. With this structure, the image display surface of the display device 1 and the surface of the retroreflector 2 are perpendicular to the direction of travel of the image light reflected from the polarization separating member 101B, simplifying the angular relationships of the surfaces constituting the optical system. If the surface of the transparent member 100 is configured to be orthogonal to the direction of travel of the image light reflected from the polarization separating member 101B, the angular relationships of the surfaces constituting the optical system can be further simplified. Figure 2C In this structure, when a user views from the direction of arrow A, the spatially suspended image 3 can be seen as a bright image. However, when other people view from the direction of arrow B, the spatially suspended image 3 cannot be seen as an image at all. This feature is very suitable for systems that display images requiring high security, and systems that display highly confidential images that need to be kept secret from people facing the user.

[0159] As explained above, Figure 2C The optical system is with Figure 2A and Figure 2B Optical systems with different structures, but capable of operating with... Figure 2A and Figure 2B The optical system also produces excellent spatial levitation images. Furthermore, it simplifies the angles of the surfaces that constitute the optical system.

[0160] Additionally, an absorptive polarizer can be provided on the surface of the transparent component 100 on the side of the polarization separation member 101B. This absorptive polarizer can be one that allows the polarized transmission of image light from the polarization separation member 101B while absorbing polarizations that are 90° out of phase with the image light from the polarization separation member 101B. This allows for sufficient transmission of image light used to form the spatial levitation image 3, while reducing external light incident from the spatial levitation image 3 side of the transparent component 100 by approximately 50%. Consequently, the amount of light generated by external light incident from the spatial levitation image 3 side of the transparent component 100 can be reduced. Figure 2C Stray light within the optical system.

[0161] <Example 3 of other structural examples of the optical system of a spatial levitation image display device>

[0162] For other structural examples of the optical system of a spatial levitation image display device, using Figure 2D Please provide an explanation. Figure 2D The optical system uses the same optical system as Figures 2A-2CThe optical system of the retroreflector 2 used is different from that of the retroreflector 5. The following uses... Figures 2D to 2I Example 3 will be described in more detail for other structures of the optical system. Figure 2D The text is marked with the same meaning. Figure 2A The structure of the same reference numerals in ~C has the same... Figures 2A-2C Same function and structure. To simplify the explanation, repeated descriptions of such structures are omitted.

[0163] Figure 2D This diagram illustrates an example of the main structure and retroreflective section structure of a spatial levitation image display device according to an embodiment of the present invention. A display device 10 for emitting image light is provided in the oblique direction of a transparent component 100 such as glass. The display device 10 includes a liquid crystal display panel 11 and a light source device 13 for generating light.

[0164] The main ray 9020 representing the light beam emitted from the display device 10 travels toward the retroreflector 5 and is incident on the retroreflector 5 at an incident angle α. The incident angle α can be, for example, 45°. However, the incident angle α is not limited to 45°, and for example, 45°±15° can be used.

[0165] The retroreflector 5 is an optical component that has the optical property of causing retroreflective reflection of light in at least a portion of its directions. Furthermore, the reflected light possesses optical properties capable of forming an image, so the retroreflector 5 can also be described as an imaging optical component or an imaging optical plate.

[0166] The specific structure of the return reflector 5 will use Figure 2E , Figure 2F As will be described in detail, under the action of the retroreflector 5, the principal ray 9020 travels in the z direction while undergoing retroreflection in the x and y directions. As a result, the reflected ray 9021 travels away from the retroreflector 5 in a mirror-symmetric path relative to the principal ray 9020 with the retroreflector 5 as the reference, and passes through the transparent component 100 to form a spatially suspended image 3 as a real image on the imaging surface.

[0167] The light beam that forms the spatial levitation image 3 is a collection of light rays converged from the retroreflector 5 to the optical image of the spatial levitation image 3. These light rays continue to travel in a straight line after passing through the optical image of the spatial levitation image 3. Therefore, the spatial levitation image 3, unlike diffused images formed on a screen using a conventional projector, is a highly directional image. Thus, in the structure shown in Figure 2, when a user views from the direction of arrow A, the spatial levitation image 3 can be seen as a bright image. However, when other people view from the direction of arrow B, the spatial levitation image 3 cannot be seen at all. This characteristic is suitable for systems displaying images requiring high security, and systems displaying highly confidential images that need to be kept secret from people directly facing the user.

[0168] use Figure 2E , Figure 2F An example of the structure of the retroreflector 5 will be described. The retroreflector 5 is a structure in which multiple corner reflectors 9040 are arranged in an array on the surface of the transparent component 50. It can also be referred to as a corner reflector array or a multi-faceted reflector array. The specific structure of the corner reflectors 9040 will be described using... Figure 2G , Figure 2H , Figure 2I In detail, the light rays 9111, 9112, 9113, and 9114 emitted from the light source 9110 are reflected twice by the two mirrors 9041 and 9042 of the corner reflector 9040, becoming reflected light rays 9121, 9122, 9123, and 9124. These two reflections are retrograde reflections in the x and y directions, returning in the same direction as the incident direction (traveling in a direction rotated 180°). In the z direction, they are normal reflections (specular reflection) due to total internal reflection, where the angle of incidence and the angle of reflection are the same.

[0169] That is, rays 9111-9114 produce reflected rays 9121-9124 along a straight line symmetrical about the corner reflector 9040 in the z-direction, forming a real image 9120 in the air. Furthermore, rays 9111-9114 emanating from the light source 9110 represent four rays of diffused light from the light source 9110. Depending on the diffusion characteristics of the light source 9110, the rays incident on the retroreflector 5 are not limited to these, but all incident rays undergo the same reflection, forming a real image 9120 in the air. Additionally, for ease of viewing, the positions of the light source 9110 and the real image 9120 in the x-direction are shown offset, but in reality, the positions of the light source 9110 and the real image 9120 in the x-direction are the same, coinciding when viewed from the z-direction.

[0170] Next, using Figure 2G , Figure 2H , Figure 2IThe structure and effect of the corner reflector 9040 constituting the retroreflector 5 will be explained. The corner reflector 9040 is a cuboid with only two specific surfaces being mirror surfaces 9041 and 9042, and the other four surfaces being transparent components. The retroreflector 5 is configured such that the corner reflectors 9040 are arranged in an array with the corresponding mirror surfaces facing the same direction.

[0171] When viewed from the top surface (+z direction), the light ray 9111 emitted from the light source 9110 is incident on the mirror 9041 (or mirror 9042) at a specific angle of incidence. After total internal reflection occurs at the reflection point 9130, total internal reflection occurs again at the reflection point 9132 on the mirror 9042 (or mirror 9041).

[0172] Let θ be the angle of incidence of ray 9111 on mirror 9041 (or mirror 9042). Then, the angle of incidence of the first reflected ray 9131, after reflection on mirror 9041 (or mirror 9042), relative to mirror 9042 (or mirror 9041), can be expressed as 90°-θ. Therefore, relative to ray 9111, the second reflected ray 9121 rotates by 2θ due to the first reflection and by 2×(90°-θ) due to the second reflection, resulting in a total reversal of 180°. On the other hand, when viewed from the side (the midway between -x and -y), total internal reflection in the z-direction occurs only once. Therefore, if φ is the angle of incidence on mirror 9041 or mirror 9042, then the reflected ray 9121 rotates by 2×φ relative to ray 9111 due to the first reflection.

[0173] As described above, the light incident on the corner reflector 9040 undergoes retrograde reflection in the x and y directions, resulting in a reversed light path, and in the z direction, it undergoes orthogonal reflection caused by total internal reflection. Considering the retrograde reflector 5, since all light paths undergo the same reflection, a point image is formed symmetrically in the z-axis direction due to the converging reversed light paths in the x and y directions.

[0174] Here, Figures 2A-2C In the optical system, the retroreflector 2 has retroreflective properties in three axes. Therefore, when a diffusing incident beam is incident on the retroreflector 2, the converging reflected beam travels relative to the retroreflector 2 towards the side where the incident light source is located. This converging reflected beam forms an image in the air, creating a spatially suspended image 3. The direction of travel of the principal ray of the converging reflected beam reflected from the retroreflector 2 is the opposite of the direction of travel of the principal ray of the diffusing incident beam incident on the retroreflector 2.

[0175] In contrast, Figure 2DIn the optical system, the retroreflector 5 has retroreflective properties along two axes and orthographic reflection along the other axis. Thus, when a diffusing incident beam is incident on the retroreflector 5, the converging reflected beam, after being reflected by the corner reflector array, travels relative to the retroreflector 5 on the opposite side from the side where the incident light source is located. This converging reflected beam forms an image in the air, creating a spatially suspended image 3.

[0176] The direction of travel of the principal ray of the converging reflected beam after being reflected by the corner reflector array of the retroreflector 5 is not the opposite direction of travel of the principal ray of the diffusing incident beam incident on the retroreflector 5. The component of the normal direction of the principal ray of the diffusing incident beam incident on the retroreflector 5 and the component of the normal direction of the surface of the plate shape of the retroreflector 5 of the direction of travel of the principal ray after being reflected on the retroreflector 5 to become a converging reflected beam do not change before and after reflection by the corner reflector array, but travel in a straight line.

[0177] That is, through reflection on the retroreflector 5, the diffuse incident beam is transformed into a converging reflected beam, but in the direction normal to the surface of the retroreflector 5, the beam travels through the retroreflector 5. Here, the diffuse incident beam incident on the retroreflector 5 and the converging reflected beam exiting the retroreflector 5 have a geometrically symmetrical relationship with respect to the surface of the retroreflector 5.

[0178] Regarding the spatial levitation image obtained by imaging the light from the image output unit 10, its resolution depends not only on the resolution of the liquid crystal display panel 11, but also significantly on... Figure 2E , Figure 2F The diameter D and spacing P of the retroreflective portion of the retroreflective plate 5 shown are not illustrated. For example, when using a 7-inch WUXGA (1920×1200 pixels) LCD panel, even if one pixel (one triplet) is approximately 80μm, if the diameter D of the retroreflective portion is 240μm and the spacing P is 300μm, then one pixel of the spatial levitation image is equivalent to 300μm. Therefore, the effective resolution of the spatial levitation image is reduced to about one-third.

[0179] Therefore, in order to make the resolution of the spatially suspended image the same as that of the display device 10, it is preferable to make the diameter D and the spacing P of the retroreflective portion close to one pixel of the liquid crystal display panel. On the other hand, in order to suppress moiré patterns caused by the pixels of the retroreflective plate and the liquid crystal display panel, the spacing ratio of each can be designed to deviate from one pixel by an integer multiple. In addition, the shape can be configured such that neither side of the retroreflective portion coincides with either side of one pixel of the liquid crystal display panel.

[0180] Furthermore, the shape of the retroreflective plate (imaging optical plate) in this embodiment is not limited to the examples described above. It can have various shapes to achieve retroreflection. Specifically, it can be various cubic corner reflectors, corner reflector arrays, slit mirror arrays, dihedral corner reflector arrays, multifaceted reflector arrays, or combinations of these reflective surfaces arranged periodically. Alternatively, a capsule lens-type retroreflective element obtained by periodically arranging glass microspheres on the surface of the retroreflective plate in this embodiment can also be used. Detailed structures of these retroreflective elements can be derived using existing technology, so detailed descriptions are omitted. Specifically, technologies disclosed in Japanese Patent Application Publication No. 2017-33005, Japanese Patent Application Publication No. 2019-133110, Japanese Patent Application Publication No. 2017-67933, and WO2009 / 131128 can be used.

[0181] in addition, Figure 2D In the optical system, the image light emitted from the display device 10 can be of any polarization state. Whether it is S-polarization or P-polarization is not a problem.

[0182] As explained above, Figure 2D Although the optical system uses the same Figures 2A-2C The optical system has a different retroreflector, but with Figures 2A-2C Similarly, its optical system is capable of creating better spatial levitation images.

[0183] Based on the above explanation Figure 2A , Figure 2B , Figure 2C , Figure 2D Its optical system can provide brighter, higher-quality images of spatial levitation.

[0184] <<Block diagram of the internal structure of the spatial levitation image display device>>

[0185] Next, a block diagram of the internal structure of the spatial levitation image display device 1000 will be described. Figure 3 This is a block diagram illustrating an example of the internal structure of a spatial levitation image display device 1000.

[0186] The spatial levitation image display device 1000 includes a retroreflective unit 1101, an image display unit 1102, a light guide 1104, a light source 1105, a power supply 1106, an external power input interface 1111, an operation input unit 1107, a non-volatile memory 1108, a memory 1109, a control unit 1110, an image signal input unit 1131, an audio signal input unit 1133, a communication unit 1132, an air operation detection sensor 1351, an air operation detection unit 1350, an audio output unit 1140, a microphone 1139, an image control unit 1160, a storage unit 1170, and a camera unit 1180. Additionally, it may also include a movable media interface 1134, an attitude sensor 1113, a transmissive self-emissive image display device 1650, a second display device 1680, or a secondary battery 1112.

[0187] The various components of the spatial levitation image display device 1000 are arranged within the housing 1190. Furthermore, Figure 3 The camera unit 1180 and the air operation detection sensor 1351 shown can also be located on the outside of the housing 1190.

[0188] Figure 3 The return reflection section 1101 corresponds to Figure 2A , Figure 2B , Figure 2C The retroreflector 2. The retroreflector 1101 causes the light modulated by the image display unit 1102 to undergo retroreflection. The light reflected from the retroreflector 1101 and output to the outside of the spatial levitation image display device 1000 forms a spatial levitation image 3.

[0189] Figure 3 The image display unit 1102 corresponds to Figure 2A , Figure 2B , Figure 2C LCD display panel 11. Figure 3 The light source 1105 corresponds to Figure 2A , Figure 2B , Figure 2C The light source device 13. Figure 3 The image display unit 1102, light guide 1104, and light source 1105 correspond to Figure 2A , Figure 2B , Figure 2C Display device 1.

[0190] The image display unit 1102 is a display unit that generates an image by modulating transmitted light based on an image signal input under the control of the image control unit 1160 (described later). The image display unit 1102 corresponds to... Figure 2A , Figure 2B , Figure 2CThe liquid crystal display panel 11. For example, a transmissive liquid crystal panel is used as the image display unit 1102. Alternatively, a reflective liquid crystal panel that modulates reflected light or a DMD (Digital Micromirror Device) panel can be used as the image display unit 1102.

[0191] The light source 1105 generates light for the image display unit 1102 and is a solid-state light source such as an LED light source or a laser light source. The power supply 1106 converts AC current input from the outside via the external power input interface 1111 into DC current to power the light source 1105. Additionally, the power supply 1106 supplies necessary DC current to various parts within the spatial levitation image display device 1000. The secondary battery 1112 stores the power supplied by the power supply 1106. Furthermore, when not powered externally via the external power input interface 1111, the secondary battery 1112 powers the light source 1105 and other structures requiring power. In other words, with the secondary battery 1112 in place, the spatial levitation image display device 1000 can be used by the user even without external power.

[0192] The light guide 1104 guides the light generated by the light source 1105, directing it towards the image display unit 1102. The combination of the light guide 1104 and the light source 1105 can also be referred to as the backlight of the image display unit 1102. The light guide 1104 can be constructed primarily of glass. Alternatively, it can be constructed primarily of plastic. It can also be constructed using a reflector. Various combinations of the light guide 1104 and the light source 1105 are possible. Specific structural examples of the combination of the light guide 1104 and the light source 1105 will be described in detail later.

[0193] The air-to-air operation detection sensor 1351 is a sensor that detects the user 230's finger operation on the spatial levitation image 3. The air-to-air operation detection sensor 1351 can, for example, sense the area overlapping with the entire display area of ​​the spatial levitation image 3. Alternatively, the air-to-air operation detection sensor 1351 can also sense only the area overlapping with at least a portion of the display area of ​​the spatial levitation image 3.

[0194] As a specific example of the airborne operation detection sensor 1351, a distance sensor using non-visible light such as infrared, non-visible light laser, or ultrasound can be cited. Alternatively, the airborne operation detection sensor 1351 can also combine multiple sensors to form a system capable of detecting coordinates in a two-dimensional plane. Furthermore, the airborne operation detection sensor 1351 can be constructed from a ToF (Time of Flight) type LiDAR (Light Detection and Ranging) sensor or an image sensor.

[0195] The airborne operation detection sensor 1351 only needs to be able to sense and detect touch operations performed by the user's finger on an object displayed as a spatial levitation image 3. Such sensing can also be performed using existing technologies.

[0196] The airborne operation detection unit 1350 acquires sensing signals from the airborne operation detection sensor 1351, determines whether the user 230's finger has made contact with an object in the space-suspended image 3 based on the sensing signals, and calculates the position of contact between the user 230's finger and the object (contact position), etc. The airborne operation detection unit 1350 is constructed, for example, by a circuit such as an FPGA (Field Programmable Gate Array). In addition, some functions of the airborne operation detection unit 1350 can also be implemented in software, for example, by a spaceborne operation detection program executed by the control unit 1110.

[0197] The airborne operation detection sensor 1351 and the airborne operation detection unit 1350 can be built into the space-based levitation image display device 1000, or they can be installed separately from the space-based levitation image display device 1000. When installed separately from the space-based levitation image display device 1000, the airborne operation detection sensor 1351 and the airborne operation detection unit 1350 are configured to transmit information and signals to the space-based levitation image display device 1000 via wired or wireless communication connection paths or image signal transmission paths.

[0198] Furthermore, the airborne operation detection sensor 1351 and the airborne operation detection unit 1350 can also be installed separately. This allows for the construction of a system that can optionally add only the airborne operation detection function, using the space-based levitation image display device 1000, which does not have airborne operation detection functionality, as the main body. Alternatively, only the airborne operation detection sensor 1351 can be separated, and the airborne operation detection unit 1350 can be integrated into the space-based levitation image display device 1000. In cases where more flexibility in the placement of the airborne operation detection sensor 1351 relative to the installation location of the space-based levitation image display device 1000 is desired, the structure that separates only the airborne operation detection sensor 1351 has advantages.

[0199] The camera unit 1180 is, for example, a camera with an image sensor, that captures images of the space near the levitation image 3 and / or the user 230's face, arms, fingers, etc. Multiple camera units 1180 can be provided. By using multiple camera units 1180, or by using a camera unit with a depth sensor, the air operation detection unit 1350 can be assisted when detecting touch operations by the user 230 on the levitation image 3. The camera unit 1180 can also be provided separately from the levitation image display device 1000. When the camera unit 1180 and the levitation image display device 1000 are provided separately, it can be configured to transmit camera signals to the levitation image display device 1000 via a wired or wireless communication connection path, etc.

[0200] For example, if the airborne operation detection sensor 1351 is configured to detect whether an object has invaded the intrusion detection plane by targeting the plane (intrusion detection plane) that includes the display surface of the spatial levitation image 3, there may be situations where the airborne operation detection sensor 1351 cannot detect how far away an object (such as a user's finger) that has not invaded the intrusion detection plane is from the intrusion detection plane, or how close the object is to the intrusion detection plane.

[0201] In this case, by using depth calculation information of the object obtained from images captured by multiple camera units 1180 and depth information of the object obtained by a depth sensor, the distance between the object and the intrusion detection plane can be calculated. Then, this information and various other information, such as the distance between the object and the intrusion detection plane, are used for various display controls of the spatial levitation image 3.

[0202] Alternatively, instead of using the air operation detection sensor 1351, the air operation detection unit 1350 can detect the user 230's touch operation on the spatial levitation image 3 based on the image captured by the camera unit 1180.

[0203] Alternatively, the camera unit 1180 can capture the face of the user 230 operating the floating image 3, and the control unit 1110 can perform user 230 recognition processing. Furthermore, to determine whether someone is standing around or behind the user 230 operating the floating image 3, or whether someone is spying on the user 230's operation of the floating image 3, the camera unit 1180 can also capture images of the area including the user 230 operating the floating image 3 and the area surrounding the user 230.

[0204] The operation input unit 1107 is, for example, a signal receiving unit or an infrared receiving unit such as an operation button or a remote control, which inputs signals for operations different from the air operation (touch operation) performed by the user 230. In addition to the user 230 who performs touch operation on the spatial levitation image 3, the operation input unit 1107 can also be used for an administrator to operate the spatial levitation image display device 1000.

[0205] The image signal input unit 1131 is connected to an external image output device to input image data. The image signal input unit 1131 can be various digital image input interfaces. For example, it can be a video input interface of the HDMI (High-Definition Multimedia Interface) standard, a video input interface of the DVI (Digital Visual Interface) standard, or a video input interface of the DisplayPort standard.

[0206] Alternatively, analog video input interfaces such as analog RGB and component video can be provided. The audio signal input unit 1133 connects to an external audio output device to input audio data. The audio signal input unit 1133 can be configured as an HDMI standard audio input interface, an optical digital terminal interface, or a coaxial digital terminal interface. When using an HDMI standard interface, the video signal input unit 1131 and the audio signal input unit 1133 can be configured as an interface integrating terminals and cables. The audio output unit 1140 can output sound based on the audio data input to the audio signal input unit 1133. The audio output unit 1140 can be configured as a speaker.

[0207] Additionally, the audio output unit 1140 can also output built-in operation tones and error warning tones. Alternatively, the audio output unit 1140 can be configured to output digital signals to external devices, similar to the Audio Return Channel function specified in the HDMI standard. The microphone 1139 is a microphone that collects sound from the vicinity of the spatial floating image display device 1000 and converts it into a signal to generate an audio signal. Alternatively, the microphone can be configured to collect the voice of a user or other person (speech), and the generated audio signal can be processed by the control unit 1110 (described later) for voice recognition processing (speech recognition processing) to obtain text information from the audio signal.

[0208] The non-volatile memory 1108 stores various data used in the spatial levitation image display device 1000. The data stored in the non-volatile memory 1108 includes, for example, various operational data for displaying the spatial levitation image 3, display icons, data on objects for user operation, and layout information. The main memory 1109 stores image data displayed as the spatial levitation image 3 and control data for the device.

[0209] The control unit 1110 controls the operation of each connected component. In addition, the control unit 1110 can cooperate with the program stored in the memory 1109 to perform calculations based on information obtained from each component in the spatial levitation image display device 1000.

[0210] The communication unit 1132 communicates with external devices, external servers, etc., via a wired or wireless communication interface. When the communication unit 1132 has a wired communication interface, this interface can be configured as, for example, a LAN interface using an Ethernet standard. When the communication unit 1132 has a wireless communication interface, it can be configured as, for example, a Wi-Fi communication interface, a Bluetooth communication interface, or a 4G or 5G mobile communication interface. Through communication via the communication unit 1132, various data such as video data, image data, and audio data are sent and received.

[0211] Additionally, the removable media interface 1134 is an interface for connecting a removable recording medium (removable medium). The removable recording medium (removable medium) can be composed of semiconductor memory such as a solid-state drive (SSD), magnetic recording media such as a hard disk drive (HDD), or optical recording media such as an optical disc. The removable media interface 1134 can read various data and information, such as image data, audio data, etc., recorded on the removable recording medium. The image data and audio data recorded on the removable recording medium are output as a spatial levitation image 3 via the image display unit 1102 and the retroreflective unit 1101.

[0212] Storage unit 1170 is a storage device that records various types of data and information, such as image data, audio data, etc. Storage unit 1170 can be composed of a magnetic recording medium recording device such as a hard disk drive (HDD) or a semiconductor device memory such as a solid-state drive (SSD). In storage unit 1170, for example, various types of data and information, such as image data, audio data, etc., can be pre-recorded at the time of product shipment. Furthermore, storage unit 1170 can also record various types of data and information, such as image data, audio data, etc., obtained from external devices and external servers via communication unit 1132.

[0213] The image data and other data recorded in the storage unit 1170 are output as a spatial levitation image 3 via the image display unit 1102 and the retroreflection unit 1101. The image data and other data of the display icons and objects for user operation displayed as the spatial levitation image 3 are also recorded in the storage unit 1170.

[0214] The layout information of the display icons, objects, etc., displayed as the spatial levitation image 3, as well as various metadata information about the objects, are also recorded in the storage unit 1170. The sound data recorded in the storage unit 1170 is output as sound, for example, from the sound output unit 1140.

[0215] The image control unit 1160 performs various controls on the image signals input to the image display unit 1102. The image control unit 1160 can also be called an image processing circuit, and may be constructed from hardware such as an ASIC, FPGA, or video processor. Alternatively, the image control unit 1160 can also be called an image processing unit or an image processing unit. For example, the image control unit 1160 controls image switching, switching which image signal from the image signal stored in the memory 1109 or the image signal (image data) input to the image signal input unit 1131 is input to the image display unit 1102.

[0216] In addition, the image control unit 1160 can also be controlled to generate a superimposed image signal obtained by superimposing the image signal stored in the memory 1109 and the image signal input from the image signal input unit 1131, and input the superimposed image signal to the image display unit 1102, thereby forming a composite image as a spatial levitation image 3.

[0217] Additionally, the image control unit 1160 can also perform control, and perform image processing on the image signals input from the image signal input unit 1131 and the image signals stored in the memory 1109. Examples of image processing include scaling (enlarging, reducing, and distorting the image), brightness adjustment (changing brightness), contrast adjustment (changing the image's contrast curve), and Retinex processing (decomposing the image into light components and changing the weights of each component).

[0218] In addition, the image control unit 1160 can also perform special effects image processing on the image signal input to the image display unit 1102 to assist the user 230 in-flight operation (touch operation). The special effects image processing is performed, for example, based on the detection results of the touch operation of the user 230 by the in-flight operation detection unit 1350 and the image captured by the camera unit 1180 of the user 230.

[0219] The attitude sensor 1113 is a sensor composed of a gravity sensor, an acceleration sensor, or a combination thereof, capable of detecting the setting attitude of the spatial levitation image display device 1000. The control unit 1110 can control the operation of each connected component based on the attitude detection results of the attitude sensor 1113. For example, if the attitude is detected to be unsuitable for the user, control can be implemented to stop the display of the image on the image display unit 1102 and display an error message to the user. Alternatively, if the attitude sensor 1113 detects a change in the setting attitude of the spatial levitation image display device 1000, control can be implemented to rotate the display direction of the image displayed on the image display unit 1102.

[0220] As explained above, the spatial levitation image display device 1000 is equipped with various functions. However, the spatial levitation image display device 1000 does not need to have all of these functions; it can have any structure as long as it has the function of forming the spatial levitation image 3.

[0221] <Structural Example of a Spatial Suspended Image Display Device>

[0222] Next, a structural example of the spatial levitation image display device will be described. Regarding the layout of the constituent elements of the spatial levitation image display device in this embodiment, various layouts may exist depending on the usage configuration. Hereinafter, [details to be added] Figures 4A to 4M The layout of each item will be explained. Additionally, in Figures 4A to 4M In any of the examples, the thick lines surrounding the space levitation image display device 1000 represent an example of the housing structure of the space levitation image display device 1000.

[0223] Figure 4A This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4A The space-floating image display device 1000 shown is equipped with... Figure 2A The optical system corresponding to the optical system. Figure 4A The spatial levitation image display device 1000 shown is horizontally positioned with the side forming the spatial levitation image 3 facing upwards. That is, Figure 4A In this device, the transparent component 100 of the spatial levitation image display device 1000 is provided on the top surface of the device. The spatial levitation image 3 is formed above the surface of the transparent component 100. The light from the spatial levitation image 3 travels obliquely upwards. With the air operation detection sensor 1351 provided as shown, it is possible to detect the user 230's finger operation on the spatial levitation image 3. Furthermore, the x-direction is the left-right direction as seen by the user, the y-direction is the front-back direction (depth direction) as seen by the user, and the z-direction is the up-down direction (vertical direction). Hereinafter, Figures 4A to 4MThe definitions of the x, y, and z directions are the same in all the diagrams, so repeated explanations are omitted.

[0224] Figure 4B This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4B The space-floating image display device 1000 shown is equipped with... Figure 2A The optical system corresponding to the optical system. Figure 4B The spatial levitation image display device 1000 shown is vertically arranged such that the side forming the spatial levitation image 3 faces the front of the spatial levitation image display device 1000 (the direction of the user 230). That is, Figure 4B In this configuration, the transparent component 100 of the spatial levitation image display device is located on the front of the device (to the user 230's direction). The spatial levitation image 3 is formed on the user 230's side, compared to the surface of the transparent component 100 of the spatial levitation image display device 1000. The light from the spatial levitation image 3 travels obliquely upwards. With the air operation detection sensor 1351 arranged as shown, it is possible to detect the user 230's finger operation on the spatial levitation image 3. Here, as... Figure 4B As shown, the air-to-air operation detection sensor 1351 senses the user 230's finger from above and uses the reflection of sensing light from the user's fingernail for touch detection. Generally, fingernails have a higher reflectivity than fingertips, so this structure improves the accuracy of touch detection.

[0225] Figure 4C This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4C The space-floating image display device 1000 shown is equipped with... Figure 2B The optical system corresponding to the optical system. Figure 4C The spatial levitation image display device 1000 shown is horizontally positioned with the side forming the spatial levitation image 3 facing upwards. That is, Figure 4C In this device, the transparent component 100 of the spatial levitation image display device 1000 is disposed on the top surface of the device. The spatial levitation image 3 is formed above the surface of the transparent component 100 of the spatial levitation image display device 1000. The light from the spatial levitation image 3 travels obliquely upward. When the air operation detection sensor 1351 is disposed as shown in the figure, it is possible to detect the operation of the user 230's finger on the spatial levitation image 3.

[0226] Figure 4D This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4D The space-floating image display device 1000 shown is equipped with... Figure 2B The optical system corresponding to the optical system. Figure 4DThe spatial levitation image display device 1000 shown is vertically arranged such that the side forming the spatial levitation image 3 faces the front of the spatial levitation image display device 1000 (the direction of the user 230). That is, Figure 4D In this configuration, the transparent component 100 of the spatial levitation image display device 1000 is located on the front of the device (to the user 230's direction). The spatial levitation image 3 is formed on the user 230's side, compared to the surface of the transparent component 100. The light from the spatial levitation image 3 travels obliquely upwards. With the air-operation detection sensor 1351 arranged as shown, it is possible to detect the user 230's finger operation on the spatial levitation image 3. Here, as... Figure 4D As shown, the air-to-air operation detection sensor 1351 senses the user 230's finger from above and uses the reflection of sensing light from the user's fingernail for touch detection. Generally, fingernails have a higher reflectivity than fingertips, so this structure improves the accuracy of touch detection.

[0227] Figure 4E This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4E The space-floating image display device 1000 shown is equipped with... Figure 2C The optical system corresponding to the optical system. Figure 4E The spatial levitation image display device 1000 shown is horizontally positioned with the side forming the spatial levitation image 3 facing upwards. That is, Figure 4E In this device, the transparent component 100 of the spatial levitation image display device 1000 is disposed on the top surface of the device. The spatial levitation image 3 is formed above the surface of the transparent component 100 of the spatial levitation image display device 1000. The light of the spatial levitation image 3 travels in the upward direction. When the air operation detection sensor 1351 is disposed as shown, it is possible to detect the operation of the user 230's finger on the spatial levitation image 3.

[0228] Figure 4F This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4F The space-floating image display device 1000 shown is equipped with... Figure 2C The optical system corresponding to the optical system. Figure 4F The spatial levitation image display device 1000 shown is vertically arranged such that the side forming the spatial levitation image 3 faces the front of the spatial levitation image display device 1000 (the direction of the user 230). That is, Figure 4FIn the spatial levitation image display device 1000, the transparent component 100 is disposed on the front of the device (to the user 230). The spatial levitation image 3 is formed on the side of the user 230, compared to the surface of the transparent component 100. The light from the spatial levitation image 3 travels in the direction in front of the user. With the air operation detection sensor 1351 disposed as shown, it is possible to detect the user 230's finger operation on the spatial levitation image 3.

[0229] Figure 4G This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4G The space-floating image display device 1000 shown is equipped with... Figure 2C The optical system corresponding to the optical system. Figures 4A to 4F In the optical system of the spatial levitation image display device, the optical path of the center of the image light emitted from display device 1 lies on the yz plane. That is, in Figures 4A to 4F Within the optical system of the spatial levitation image display device, the image light travels in the front-back and up-down directions as seen from the user's perspective. In contrast, in... Figure 4G In the optical system of the spatial levitation image display device shown, the optical path of the center of the image light emitted from display device 1 lies on the xy plane. That is, in Figure 4G Within the optical system of the spatial levitation image display device shown, the image light travels in the left-right and front-back directions from the user's perspective. Figure 4G The spatial levitation image display device 1000 shown is positioned such that the side forming the spatial levitation image 3 faces the front of the device (the direction of the user 230). That is, Figure 4G In the spatial levitation image display device 1000, the transparent component 100 is located on the front of the device (to the user 230's direction). The spatial levitation image 3 is formed on the user's side, compared to the surface of the transparent component 100. The light from the spatial levitation image 3 travels in the direction in front of the user. With the air-operation detection sensor 1351 arranged as shown, it is possible to detect the user 230's finger operation on the spatial levitation image 3.

[0230] Figure 4H This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4H The spatial levitation image display device 1000 has a window made of a transparent plate 100B, such as glass or plastic, on the back of the device (the side opposite to the position where the user 230 views the spatial levitation image 3, i.e., the side opposite to the direction of travel of the image light of the spatial levitation image 3 toward the user 230). This is consistent with... Figure 4G Unlike spatial levitation image display devices, other structures are different. Figure 4G The spatial levitation image display device has the same structure, so repeated descriptions are omitted. Figure 4H In the spatial levitation image display device 1000, for the spatial levitation image 3, a window with a transparent plate 100B is provided on the opposite side of the direction of travel of the image light of the spatial levitation image 3. Therefore, when the user 230 views the spatial levitation image 3, the scenery behind the spatial levitation image display device 1000 can be identified as the background of the spatial levitation image 3. Thus, the user 230 can perceive that the spatial levitation image 3 is suspended in the air in front of the scenery behind the spatial levitation image display device 1000. This further emphasizes the sense of suspension of the spatial levitation image 3.

[0231] Furthermore, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, there is a possibility that a portion of the image light output from the display device 1 may be reflected by the polarization separation member 101B and go to the transparent plate 100B. And depending on the coating performance of the surface of the transparent plate 100B, there is a possibility that this light may be reflected again on the surface of the transparent plate 100B as stray light and be seen by the user. Therefore, to prevent this stray light, the transparent plate 100B may not be provided on the aforementioned window on the back of the spatially suspended image display device 1000.

[0232] Figure 4I This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4I Spatial levitation image display device 1000 and Figure 4H The difference in the spatial levitation image display device is that a light-blocking door 1410 is provided at the window of the transparent plate 100B located on the back of the device (on the opposite side of the position where user 230 views the spatial levitation image 3). Other structures are the same as... Figure 4H The spatial levitation image display device has the same structure, so repeated descriptions are omitted.

[0233] Figure 4I The opening / closing door 1410 of the spatial levitation image display device 1000 has, for example, a light-shielding plate, including a mechanism for moving (sliding) the light-shielding plate, a rotation mechanism, or a mechanism for making the light-shielding plate detachable. This allows the window (rear side window) of the transparent plate 100B located at the depth side of the spatial levitation image display device 1000 to switch between an open state and a light-shielding state. The opening / closing door 1410 can be driven by a motor (not shown) to electrically move (slide) and rotate the light-shielding plate. This motor can be... Figure 3 The control unit 1110 controls this. Additionally, Figure 4I The example disclosed shows that the number of light-shielding panels of the opening and closing door 1410 is 2. However, the number of light-shielding panels of the opening and closing door 1410 can also be 1.

[0234] For example, when the view visible from the depths of the window of the transparent panel 100B of the spatial levitation image display device 1000 is outdoors, the brightness of sunlight varies with the weather. When the outdoor sunlight is strong, the background of the spatial levitation image 3 becomes too bright, reducing the user 230's ability to distinguish the spatial levitation image 3. In such cases, if the back window is shaded by moving (sliding), rotating, or installing the light-shielding plate of the opening / closing door 1410, the background of the spatial levitation image 3 will darken, thus relatively improving the distinguishability of the spatial levitation image 3. This light-shielding action of the light-shielding plate of the opening / closing door 1410 can also be performed manually by the user 230. Alternatively, it can be performed via... Figure 3 The operation input is made by the operation input unit 1107, and the control unit 1110 controls the motor (not shown) to perform the light-blocking action of the light-blocking plate of the door 1410 to open and close accordingly.

[0235] Alternatively, an illuminance sensor can be installed on the rear side of the spatially suspended image display device 1000 (the opposite side of the user 230) – for example, near the rear-side window – to measure the brightness of the space outside the rear-side window. In this case, the brightness can be determined based on the detection results of the illuminance sensor. Figure 3 The control unit 1110 controls a motor (not shown) to open and close the light shield of the door 1410. By controlling the opening and closing of the light shield of the door 1410 in this way, the recognizability of the spatial floating image 3 can be better maintained even if the user 230 does not manually open and close the light shield of the door 1410.

[0236] Furthermore, the light shield of the opening / closing door 1410 can also be manually detachable. Depending on the intended use and environment of the spatial floating image display device 1000, the user can choose whether the rear window is open or blocked. If the rear window is intended to be kept blocked for an extended period, the detachable light shield can be fixed in the blocked state. Alternatively, if the rear window is intended to be kept open for an extended period, the detachable light shield can be removed. The light shield can be attached or removed using screws, hooks, or an embedded structure.

[0237] in addition, Figure 4IThe example of the spatial levitation image display device 1000 is similar. Depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, there is a possibility that a portion of the image light output from the display device 1 may be reflected on the polarization separation member 101B and go to the transparent plate 100B. Furthermore, depending on the coating properties of the surface of the transparent plate 100B, there is a possibility that the light may be reflected again on the surface of the transparent plate 100B as stray light and be seen by the user. Therefore, to prevent this stray light, the transparent plate 100B may not be provided on the aforementioned window on the back of the spatial levitation image display device 1000. The aforementioned opening / closing door 1410 may be provided on the window without the transparent plate 100B. To prevent this stray light, the inner surface of the housing of the light-shielding plate of the opening / closing door 1410 preferably has a coating or material with low light reflectivity.

[0238] Figure 4J This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4J Spatial levitation image display device 1000 and Figure 4H The difference in this spatial levitation image display device is that, instead of a transparent glass or plastic panel 100B, it uses an electrically controlled variable transmittance device 1620 for its rear side window. Other structural features are similar to... Figure 4H The structure is the same as that of a spatial levitation image display device, so repeated descriptions are omitted. An example of an electronically controlled variable transmittance device 1620 is a liquid crystal shutter, etc.

[0239] In other words, the liquid crystal shutter controls the transmission of light by voltage-controlled liquid crystal elements sandwiched between two polarizers. Therefore, if the transmittance of the liquid crystal shutter is increased, the background of the suspended image 3 becomes transparent, allowing the view through the back side window to be seen. Conversely, if the transmittance of the liquid crystal shutter is increased, the background of the suspended image 3 becomes so transparent that the view through the back side window is not visible.

[0240] Furthermore, the LCD shutter allows for control of intermediate color levels, thus enabling the use of states such as 50% transmittance. For example, it can be set according to the transmission path... Figure 3 The operation input unit 1107 performs the operation input, and the control unit 1110 controls the transmittance of the electronically controlled transmittance variable device 1620. With this structure, when viewing the scenery through the back side window as the background of the spatial levitation image 3, but the background, i.e., the scenery through the back side window, is too bright, reducing the recognizability of the spatial levitation image 3, the recognizability of the spatial levitation image 3 can be adjusted by adjusting the transmittance of the electronically controlled transmittance variable device 1620.

[0241] Alternatively, an illuminance sensor can be installed on the rear side of the spatially suspended image display device 1000 (the opposite side of the user 230) – for example, near the rear-side window – to measure the brightness of the space outside the rear-side window. In this case, the brightness can be determined based on the detection results of the illuminance sensor. Figure 3 The control unit 1110 controls the transmittance of the electronically controlled transmittance variable device 1620. Thus, even if the user 230 does not... Figure 3 The operation input unit 1107 can input operations, and the transmittance of the electronically controlled transmittance variable device 1620 can be adjusted according to the brightness of the space outside the rear side window, so that the recognizability of the spatial floating image 3 can be better maintained.

[0242] Furthermore, the example described above uses an electrically controlled variable transmittance device 1620, which is an example of a liquid crystal shutter. However, electronic paper can also be used as another example of an electrically controlled variable transmittance device 1620. Even when using electronic paper, the same effect as described above can be achieved. Moreover, the power consumption of electronic paper in maintaining intermediate color levels is very low. Therefore, compared with the case of using a liquid crystal shutter, a low-power spatial levitation image display device can be realized.

[0243] Figure 4K This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4K Spatial levitation image display device 1000 and Figure 4G The difference between this spatial levitation image display device and the previous one is that it has a transmissive self-emissive image display device 1650 instead of the transparent component 100. Other structures are the same as... Figure 4G The spatial levitation image display device has the same structure, so repeated descriptions are omitted.

[0244] exist Figure 4K In the spatial levitation image display device 1000, after the image beam passes through the display surface of the transmissive self-emissive image display device 1650, a spatial levitation image 3 is formed outside the spatial levitation image display device 1000. That is, when an image is displayed using the transmissive self-emissive image display device 1650 as a two-dimensional flat panel display, the spatial levitation image 3 can be displayed as a floating image in front of the image on the transmissive self-emissive image display device 1650. At this time, the user 230 can simultaneously see two images with different depth positions. The transmissive self-emissive image display device 1650 can be constructed using existing technologies such as the transmissive organic EL panel disclosed in Japanese Patent Application Publication No. 2014-216761. In addition, the transmissive self-emissive image display device 1650 in Figure 3 Not shown in the image, but can be used as... Figure 3 It is a component of the spatial levitation image display device 1000 and is connected to other processing units such as the control unit 1110.

[0245] This allows for the display of both background and character objects on the transmissive self-emissive image display device 1650, followed by the movement of only the character objects to the suspended image 3 in front of the user. This provides the user 230 with a more effective "surprise effect" image experience.

[0246] Furthermore, if the interior of the spatial levitation image display device 1000 is in a light-blocking state, the background of the transmissive self-emissive image display device 1650 is sufficiently dark. Therefore, when the display device 1 does not display an image or the light source of the display device 1 does not emit light, and only the transmissive self-emissive image display device 1650 displays an image, for the user 230, the transmissive self-emissive image display device 1650 appears not to be a transmissive display but to be a typical two-dimensional flat panel display (in the embodiments of the present invention, the spatial levitation image 3 is displayed as a real optical image in a space where there is no screen, so if the light source of the display device 1 is not emitted, the predetermined display position of the spatial levitation image 3 is a space where nothing exists). Therefore, by using the transmissive self-emissive image display device 1650 to display images as if it were a typical two-dimensional flat panel display, and then suddenly displaying characters and objects in the air as spatial levitation images 3, a more effective "surprise effect" image experience can be provided to the user 230.

[0247] Furthermore, the darker the interior of the spatial levitation image display device 1000, the more the transmissive self-emissive image display device 1650 appears to be a two-dimensional flat panel display. Therefore, an absorptive polarizer (not shown) can be provided on one side of the spatial levitation image display device 1000 (the incident surface where the image light reflected from the polarization separation member 101B is incident on the transmissive self-emissive image display device 1650, i.e., the side of the transmissive self-emissive image display device 1650 opposite to the spatial levitation image 3) to transmit the polarization of the image light reflected from the polarization separation member 101B and absorb the polarization that is 90° out of phase with the polarization. In this way, the impact on the image light forming the spatial levitation image 3 is not significant, but the light incident from the outside through the transmissive self-emissive image display device 1650 into the interior of the spatial levitation image display device 1000 can be greatly reduced, making the interior of the spatial levitation image display device 1000 darker, which is preferable.

[0248] Figure 4L This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4L The space-floating image display device 1000 is Figure 4K A variation of the space-suspended image display device. The configuration orientation of the structure in the space-suspended image display device 1000 is... Figure 4K Unlike spatial levitation image display devices, it is different from... Figure 4F The configuration is similar to that of a spatial levitation image display device. Regarding the functions and operations of each structure, because... Figure 4K The spatial levitation image display device has the same structure, so repeated descriptions are omitted.

[0249] Figure 4L The same applies to the spatial levitation image display device. After the beam of image light passes through the transmissive self-emissive image display device 1650, a spatial levitation image 3 is formed on the side of the transmissive self-emissive image display device 1650 closer to the user 230.

[0250] Whether Figure 4K Examples of spatial levitation image display devices are still Figure 4L In the example of the spatial levitation image display device, from the user's perspective (230), the spatial levitation image 3 is superimposed on the image of the transmissive self-emissive image display device 1650 (on the side in front of the user). Here, the position of the spatial levitation image 3 differs from the position of the image of the transmissive self-emissive image display device 1650 in the depth direction. Therefore, when the user moves their head (viewpoint), they can perceive the depth of the two images due to parallax. Thus, by displaying two images with different depth positions, a better naked-eye 3D image experience can be provided to the user without the need for stereoscopic glasses.

[0251] Figure 4M This diagram illustrates an example of the structure of a spatially suspended image display device. Figure 4M In the spatial levitation image display device 1000, relative to Figure 4G The polarization separation component 101B of the spatial levitation image display device has a second display device 1680 located on the depth side as viewed from the user. Other structures are similar to... Figure 4G The spatial levitation image display device has the same structure, so repeated descriptions are omitted.

[0252] Figure 4M In the illustrated structural example, the second display device 1680 is disposed at a depth of the display position of the spatial levitation image 3, with its image display surface facing the spatial levitation image 3. With this structure, from the user 230's perspective, the image displayed on the second display device 1680 and the spatial levitation image 3, both displayed at different depths, can be seen overlappingly. That is, it can be considered that the second display device 1680 is positioned in the direction of displaying the image facing the user 230 who is viewing the spatial levitation image 3. Furthermore, the second display device 1680... Figure 3 Not shown in the image, but can be used as... Figure 3 It is a component of the spatial levitation image display device 1000 and is connected to other processing units such as the control unit 1110.

[0253] in addition, Figure 4M The image light from the second display device 1680 of the spatial levitation image display device 1000 is viewed by the user 230 after passing through the polarization separation member 101B. Therefore, in order to better transmit the image light from the second display device 1680 through the polarization separation member 101B, it is preferable that the image light output from the second display device 1680 is light with a polarization direction more suitable for transmission through the polarization separation member 101B. That is, it is preferable that the polarization direction is the same as that of the image light output from the display device 1. For example, if the image light output from the display device 1 is S-polarized light, it is preferable that the image light output from the second display device 1680 is also S-polarized light. Similarly, if the image light output from the display device 1 is P-polarized light, it is preferable that the image light output from the second display device 1680 is also P-polarized light.

[0254] Figure 4M The example of a spatial levitation image display device also displays a second image at a depth of the spatial levitation image 3, which in this respect has the same characteristics as... Figure 4K Examples of spatial levitation image display devices and Figure 4L The same effect as the example of a spatially suspended image display device. However, with Figure 4K Examples of spatial levitation image display devices and Figure 4L Unlike examples of spatial levitation image display devices, in Figure 4M In the example of the spatial levitation image display device, the beam of image light used to form the spatial levitation image 3 does not pass through the second display device 1680. Therefore, the second display device 1680 does not need to be a transmissive self-emissive image display device; it can be a liquid crystal display (LCD) as a two-dimensional flat panel display. The second display device 1680 can also be an organic EL display. Thus, with... Figure 4K Examples of spatial levitation image display devices and Figure 4L Compared to examples of spatially suspended image display devices. Figure 4M The example of a spatial levitation image display device enables the realization of the spatial levitation image display device 1000 at a lower cost.

[0255] Here, depending on the polarization distribution of the image light output from display device 1 and the performance of polarization separation component 101B, there is a possibility that a portion of the image light output from display device 1 may be reflected on polarization separation component 101B and go to second display device 1680. There is a possibility that this light (a portion of the image light) may be reflected again on the surface of second display device 1680 as stray light and be seen by the user.

[0256] Therefore, to prevent stray light, an absorptive polarizer can be provided on the surface of the second display device 1680. In this case, the absorptive polarizer can be an absorptive polarizer that transmits polarized image light output from the second display device 1680 and absorbs polarized light with a polarization 90° out of phase with the image light output from the second display device 1680. Furthermore, when the second display device 1680 is a liquid crystal display (LCD), an absorptive polarizer is also present on the image emission side inside the LCD. However, if the emission surface of the absorptive polarizer on the image emission side inside the LCD has a glass cover (a glass cover on the image display side), it will be impossible to prevent stray light generated by reflection of light from outside the LCD from the glass cover. Therefore, it is necessary to additionally provide the aforementioned absorptive polarizer on the surface of the glass cover.

[0257] Furthermore, when displaying images using the second display device 1680, which is a two-dimensional flat panel display, the spatially suspended image 3 can be displayed as an image on the side of the image on the second display device 1680 closer to the user (the side in front of the user). At this time, the user 230 can simultaneously see two images with different depth positions. By displaying the character using the spatially suspended image 3 and displaying the background on the second display device 1680, the user 230 can be provided with an effect as if they can see the space where the character is located in three dimensions.

[0258] Another effect can be achieved by displaying both the background and the character on the second display device 1680, and then moving only the character and the other objects to the space in front of the user to float the image 3. This can provide the user 230 with a more effective "surprise effect" image experience.

[0259] then, Figure 4N This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4N The space-floating image display device 1000 adopts Figure 2D A spatial levitation image display device with an optical system. (And employing...) Figures 2A-2C Similarly, in the example of a spatial levitation image display device with an optical system, an image light passing through the transparent component 100 is used to image a spatial levitation image 3 in the air. In addition, the user's finger 9004 can be used to detect the operation of the spatial levitation image 3 by using the sensing light of the air operation detection sensor 1351, which is located deep inside the transparent component 100 from the user's perspective.

[0260] Regardless of what was adopted Figures 2A-2C In the example of a space-suspended image display device using an optical system, it still employs... Figure 2DIn the example of the spatial levitation image display device with an optical system, the spatial levitation images 3 are all imaged in front of the transparent component 100 (on the side closest to the user). The user's finger operation on the spatial levitation image 3 can be detected using the sensing light of the air operation detection sensor 1351, which is located deep within the transparent component 100 from the user's perspective. Therefore, when using... Figure 2D In the spatial levitation image display device of the optical system, a [device] is arranged deep within the transparent component 100 as seen from the user. Figures 2A-2C The optical system is different from that of a spatial levitation image display device with an optical system.

[0261] However, in terms of ease of use from the user's perspective, it adopted... Figure 2D The space-suspended image display device with optical system has the same characteristics as the one using optical system. Figures 2A-2C The optical system has roughly the same ease of use as the space-suspended image display device.

[0262] then, Figure 4O This is a diagram illustrating an example of the structure of a spatially suspended image display device. Figure 4O Is Figure 4N The diagram shows the structure of the internal optical system in the spatial levitation image display device 1000. Figure 4O The space-floating image display device 1000 shown is equipped with... Figure 2D The optical system corresponding to the optical system. Figure 4O The spatial levitation image display device 1000 shown is horizontally arranged with the side forming the spatial levitation image 3 facing upwards.

[0263] Right now, Figure 4O In this device, the transparent component 100 of the spatial levitation image display device 1000 is disposed on the top surface of the device. The spatial levitation image 3 is formed above the surface of the transparent component 100 of the spatial levitation image display device 1000. The light from the spatial levitation image 3 travels obliquely upward. When the air operation detection sensor 1351 is disposed as shown in the figure, it is possible to detect the operation of the user 230's finger on the spatial levitation image 3.

[0264] Here, for Figure 4O Structure and Figure 4A The structures were compared to confirm their differences. Figure 4A In this configuration, the display device 1 and the suspended spatial image 3 are symmetrical about the plane of the polarization separation component 101. In contrast, Figure 4O In this arrangement, the display device 1 and the spatially suspended image 3 are symmetrical about the surface of the retroreflector 5. Furthermore, Figure 4A The structure contains a retroreflector 2 and a λ / 4 waveplate 21, but Figure 4O They do not exist in [the text]. Furthermore, Figure 4A More preferably, it has an absorptive polarizer 12, but Figure 4O There is no special need for an absorptive polarizer 12.

[0265] That is, in order to Figure 4A In the structure Figure 2A The optical system was replaced with Figure 2D The optical system was thus replaced with Figure 4O The structure can be manipulated as follows. That is, by... Figure 4A The polarization separation component 101 in the structure is replaced with a retroreflector 5, from Figure 4A The structure can be modified by removing the retroreflector 2 and the λ / 4 waveplate 21. The absorptive polarizer 12 is optional. By making substitutions based on this concept, it is possible to... Figures 4A-4G The structure of the space-suspended image display device is equipped with Figures 2A-2C The optical system was replaced with Figure 2D The optical system was thus replaced with one equipped with Figure 2D A spatial levitation image display device with an optical system. At this time, in Figure 4A and Figure 4B The polarization separation component 101 can be replaced with the retroreflector 5. Figures 4C to 4G The polarization separation component 101B can be replaced with the retroreflector 5.

[0266] In this way, it can be achieved in Figures 4A-4G The optical system is replaced in the structure of the spatial levitation image display device. Figure 2D A spatial levitation image display device derived from an optical system. In these replacements... Figure 2D In the space-based levitation image display device of the optical system, it is also possible to achieve [interaction / cooperation]. Figures 4A-4G The spatial levitation image display device has roughly the same ease of use as the spatial levitation image display device.

[0267] <Display Device>

[0268] Next, the display device 1 of this embodiment will be described using the accompanying drawings. The display device 1 of this embodiment includes an image display element 11 (liquid crystal display panel) and a light source device 13 constituting its light source. Figure 5 In the figure, the light source device 13 is shown in an unfolded perspective view along with the liquid crystal display panel.

[0269] The liquid crystal display panel (image display element 11) is as follows Figure 5As indicated by arrow 30, an illumination beam with characteristics similar to laser light—namely, strong directional (straight-line travel) and a uniform polarization plane—is received from the light source device 13, which serves as a backlight device. The liquid crystal display panel (image display element 11) modulates the received illumination beam according to the input image signal. The modulated image light is reflected by the retroreflector 2 and passes through the transparent component 100 to form a spatially suspended image of a real image (see reference). Figure 1 ).

[0270] in addition, Figure 5 The display device 1 includes a liquid crystal display panel 11 constituting the display device 1, and a light direction conversion panel 54 that controls the pointing characteristics of the emitted light beam from the light source device 13. It also includes a narrow-angle diffuser (not shown) as needed. Specifically, polarizers are provided on both sides of the liquid crystal display panel 11, and image light with a specific polarization is emitted according to the intensity of the light modulated by the image signal (see reference). Figure 5 Arrow 30). Thus, the desired image, as highly directional (straight-line travel) polarized light, is projected onto the retroreflector 2 via the light direction conversion panel 54. After being reflected on the retroreflector 2, it is transmitted and reaches the viewer's eye outside the shop (space), forming a spatially suspended image 3. Alternatively, a protective cover 50 can be provided on the surface of the aforementioned light direction conversion panel 54 (see arrow 30). Figure 6 , Figure 7 ).

[0271] <Example 1 of a display device>

[0272] Figure 6 An example illustrating the specific structure of display device 1. Figure 6 exist Figure 5 The light source device 13 is equipped with a liquid crystal display panel 11 and a light direction conversion panel 54. The light source device 13... Figure 5 The housing shown is made of materials such as plastic, and contains LED components 201 and a light guide 203. At the end face of the light guide 203, in order to... Figure 5 The lens shape shown converts the divergent light from each LED element 201 into a roughly parallel beam. It has a cross-sectional area that gradually increases towards the surface opposite the light-receiving part, and it allows for multiple total internal reflections during internal propagation, thereby gradually reducing the divergence angle. A liquid crystal display panel 11 constituting the display device 1 is mounted on the upper surface of the display device 1. Furthermore, an LED substrate 202 is mounted on one side (in this example, the left end face) of the housing of the light source device 13, on which LED (Light Emitting Diode) elements 201, serving as semiconductor light sources, and their control circuitry are mounted. A heat sink, used to cool the heat generated in the LED elements and control circuitry, can be mounted on the outer surface of the LED substrate 202.

[0273] Furthermore, on the frame (not shown) of the liquid crystal display panel mounted on the upper surface of the housing of the light source device 13, a liquid crystal display panel 11 mounted on the frame, and an FPC (Flexible Printed Circuits) (not shown) electrically connected to the liquid crystal display panel 11 are mounted. That is, the liquid crystal display panel 11, as an image display element, and the LED element 201, as a solid-state light source, are based on the control circuit constituting the electronic device (…). Figure 3 The image control unit 1160 modulates the intensity of the transmitted light using a control signal to generate a display image. At this time, the generated image light has a narrow diffusion angle and only a specific polarization component, thus approximating a surface-emitting laser image source driven by an image signal, resulting in a novel image display device unlike any previously known. Furthermore, currently, obtaining a laser beam of the same size as the image obtained by the aforementioned display device 1 using a laser device is technically and safety-wise impossible. Therefore, in this embodiment, for example, a beam emitted by a conventional light source having LED elements is used to obtain light that approximates the surface-emitting laser image light.

[0274] Next, the structure of the optical system housed within the housing of the light source device 13 will be described with reference to... Figure 6 as well as Figure 7 A detailed explanation will follow.

[0275] because Figure 6 and Figure 7 This is a cross-sectional view, so only one of the multiple LED elements 201 constituting the light source is shown. They are transformed into approximately collimated light by the shape of the light-receiving end face 203a of the light guide 203. Therefore, the light-receiving part of the light guide end face is mounted in a predetermined positional relationship with the LED element.

[0276] Furthermore, the light guide 203 is formed, for example, using a light-transmitting resin such as acrylic resin. The LED light-receiving surface at the end of the light guide 203 has, for example, an outer peripheral surface in the shape of a convex cone obtained by rotating a parabolic section. Its top has a recess, and a convex portion (i.e., a convex lens surface) is formed in the center of this recess. Furthermore, a convex lens surface protruding outwards (or a concave lens surface recessed inwards) is formed in the center of its flat portion (not shown). Additionally, the light-receiving portion of the light guide on which the LED element 201 is mounted has a parabolic shape forming a conical outer peripheral surface, and is set within an angle range that allows light emitted from the LED element in the peripheral direction to be totally internally reflected, or a reflective surface is formed.

[0277] On the other hand, LED elements 201 are respectively disposed at predetermined positions on the surface of its circuit board, i.e., LED substrate 202. The LED substrate 202 is disposed and fixed relative to the LED collimator (light-receiving end face 203a) such that the LED elements 201 on its surface are respectively located at the center of the aforementioned recess.

[0278] According to this structure, the shape of the light-receiving end face 203a of the light guide 203 can be used to make the light emitted from the LED element 201 become approximately parallel light and output, thereby improving the utilization efficiency of the generated light.

[0279] As described above, the light source device 13 is constructed by mounting a light source unit consisting of a plurality of LED elements 201 arranged as light sources at the light-receiving end face 203a, which is provided on the end face of the light guide 203. For the diverging light beam from the LED elements 201, the light-receiving end face 203a of the light guide end face makes it approximately parallel light, and guides the light inside the light guide 203 (in a direction parallel to the paper surface) as shown by the arrow. The beam direction conversion unit 204 directs the light beam to be emitted towards the liquid crystal display panel 11 (in a direction perpendicular to the paper surface) which is arranged approximately parallel to the light guide 203. By optimizing the distribution (density) of the beam direction conversion unit 204 with the shape of the inside or surface of the light guide, the uniformity of the light beam incident on the liquid crystal display panel 11 can be controlled.

[0280] The aforementioned beam direction conversion unit 204, by utilizing the shape of the light guide surface or by providing portions with different refractive indices within the light guide, directs the beam propagating within the light guide towards the liquid crystal display panel 11 (in a direction perpendicular to the paper surface) arranged approximately parallel to the light guide 203. At this time, for the liquid crystal display panel 11, with the viewpoint positioned directly opposite the center of the screen and the diagonal dimension of the screen aligned with the center, the brightness of the center and periphery of the screen is compared. A relative brightness ratio of 20% or higher is sufficient for practical use, and a ratio exceeding 30% indicates even better performance.

[0281] in addition, Figure 6 This is a cross-sectional view illustrating the structure and function of the light source in the light source device 13, which includes the light guide 203 and the LED element 201, according to this embodiment. Figure 6 In this light source device 13, for example, there is a light guide 203 made of plastic or the like with a beam direction conversion unit 204 provided on its surface or inside, an LED element 201 as a light source, a reflector 205, a phase difference plate 206, a cylindrical lens, etc., and a liquid crystal display panel 11 with polarizers on the light source incident surface and the image light exit surface is mounted on its upper surface.

[0282] Furthermore, a thin film or sheet-like reflective polarizer 49 is provided on the light incident surface (lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, so that one polarization (e.g., P-light) 212 of the natural light beam 210 emitted from the LED element 201 is selectively reflected. The reflected light is reflected again by a reflective sheet 205 provided on one surface (lower surface in the figure) of the light guide 203 and goes to the liquid crystal display panel 11. Therefore, a phase retardation plate (λ / 4 waveplate) is provided between the reflective sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizer 49, so that the light is reflected on the reflective sheet 205 and passes through the phase retardation plate twice, thereby changing the reflected beam from P-polarization to S-polarization, improving the utilization efficiency of the light source light as image light. The light intensity is modulated by the liquid crystal display panel 11 according to the image signal of the image beam ( Figure 6 Arrow 213) is incident on the retroreflector 2. After reflection on the retroreflector 2, a spatially suspended image of a real image can be obtained.

[0283] Figure 7 Is with Figure 6 Similarly, a cross-sectional view is used to illustrate the structure and function of the light source in this embodiment, which performs polarization transformation in the light source device 13 including the light guide 203 and the LED element 201. The light source device 13 also similarly includes, for example, a light guide 203 made of plastic or the like with a beam direction transformation unit 204 provided on its surface or internally, an LED element 201 serving as the light source, a reflector 205, a phase retardation plate 206, a cylindrical lens, etc. A liquid crystal display panel 11, having polarizers on the light source incident surface and the image light exit surface, is mounted as an image display element on the upper surface of the light source device 13.

[0284] A thin film or sheet-like reflective polarizer 49 is provided on the light incident surface (lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, so that a certain polarization (e.g., S-ray) 211 in the natural light beam 210 emitted from the LED element 201 is selectively reflected. That is, Figure 7 In the example, the selective reflection characteristics of the reflective polarizer 49 are similar to... Figure 7 The reflected light is reflected by a reflective sheet 205 located on one surface of the light guide 203 (lower in the figure) and then travels back to the liquid crystal display panel 11. A phase retardation plate (λ / 4 waveplate) is provided between the reflective sheet 205 and the light guide 203, or between the light guide 203 and the reflective polarizer 49. The light is then reflected by the reflective sheet 205 and passes through the phase retardation plate twice, thereby changing the reflected beam from S-polarization to P-polarization and improving the utilization efficiency of the light source as image light. The light intensity is modulated by the liquid crystal display panel 11 according to the image signal of the image beam (…). Figure 7 Arrow 214) is incident on the retroreflector 2. After reflection on the retroreflector 2, a spatially suspended image of a real image can be obtained.

[0285] exist Figure 6 and Figure 7 In the light source device shown, besides the function of the polarizer on the light incident surface of the corresponding liquid crystal display panel 11, the theoretically achievable contrast ratio is the product of the reciprocal of the orthogonal transmittance of the reflective polarizer and the reciprocal of the orthogonal transmittance obtained from the two polarizers attached to the liquid crystal display panel, since a polarization component is reflected by a reflective polarizer. Therefore, a high contrast ratio can be obtained. In fact, experiments have confirmed that the contrast ratio of the displayed image is improved by more than 10 times. As a result, high-quality images comparable to those obtained with self-emissive organic EL displays can be obtained.

[0286] <Example 2 of a display device>

[0287] Figure 8 This is another example of the specific structure of the display device 1. The light source device 13 is constructed by housing LEDs, collimators, composite diffusers, light guides, etc., in a housing such as plastic, and a liquid crystal display panel 11 is mounted on its upper surface. In addition, an LED substrate is mounted on one side of the housing of the light source device 13, on which LED (Light Emitting Diode) elements 14a and 14b, which serve as semiconductor light sources, and their control circuits are mounted. A heat sink 103, which is used to cool the heat generated in the LED elements and the control circuits, is mounted on the outer side of the LED substrate.

[0288] Furthermore, a liquid crystal display panel 11 mounted on the upper surface of the housing, and an FPC (Flexible Printed Circuits) 403 electrically connected to the liquid crystal display panel 11 are provided on the liquid crystal display panel frame. That is, the liquid crystal display panel 11, as a liquid crystal display element, and the LED elements 14a and 14b, as solid-state light sources, together modulate the intensity of transmitted light based on control signals from the control circuit (not shown here) constituting the electronic device to generate a display image.

[0289] <Example 3 of a display device>

[0290] Next, use Figure 9Another example illustrating the specific structure of display device 1 (Example 3 of the display device). The light source device of this display device 1 uses a collimator 18 to transform the divergent beam of light (a mixture of P-polarized and S-polarized light) from the LED into a substantially parallel beam, and uses the reflective surface of the reflective light guide 304 to reflect it towards the liquid crystal display panel 11. The reflected light is incident on a reflective polarizer 49 disposed between the liquid crystal display panel 11 and the reflective light guide 304. The reflective polarizer 49 allows light of a specific polarization (e.g., P-polarized light) to pass through, causing the transmitted polarized light to enter the liquid crystal display panel 11. Here, other polarizations (e.g., S-polarized light) are reflected by the reflective polarizer 49 and return to the reflective light guide 304.

[0291] The reflective polarizer 49 is tilted relative to the liquid crystal display panel 11 in a manner that is not perpendicular to the principal ray of light from the reflective surface of the reflective light guide 304. The principal ray of light reflected from the reflective polarizer 49 is incident on the transmissive surface of the reflective light guide 304. Light incident on the transmissive surface of the reflective light guide 304 passes through the back surface of the reflective light guide 304 and is reflected on the reflective plate 271 by the λ / 4 waveplate 270, which serves as a phase retardation plate. The light reflected on the reflective plate 271 passes through the λ / 4 waveplate 270 again and passes through the transmissive surface of the reflective light guide 304. The light passing through the transmissive surface of the reflective light guide 304 is then incident on the reflective polarizer 49 again.

[0292] At this point, the light that is incident on the reflective polarizer 49 again passes through the λ / 4 waveplate 270 twice, so its polarization is transformed into a polarization that can pass through the reflective polarizer 49 (e.g., P-polarization). Thus, the polarization-transformed light passes through the reflective polarizer 49 and enters the liquid crystal display panel 11. Alternatively, the polarization design in the polarization transformation can be reversed compared to the above description (switching S-polarization and P-polarization).

[0293] As a result, the light from the LEDs is uniformly polarized (e.g., P-polarized) and incident on the liquid crystal display panel 11, where brightness modulation is performed accordingly based on the image signal to display an image on the panel surface. Multiple LEDs constituting the light source are shown in the same way as in the example above (but because it is a longitudinal cross-sectional view, ...). Figure 9 (Only one is shown in the figure), and they are installed in the specified positions relative to the collimator 18.

[0294] Furthermore, the collimator 18 is formed, for example, using a light-transmitting resin such as acrylic resin or glass. The collimator 18 may have an outer peripheral surface with a convex conical shape obtained by rotating a parabolic section. Additionally, at the center of the top portion of the collimator 18 (the side opposite to the LED substrate 102), a recess with a protrusion (i.e., a convex lens surface) may be formed. Furthermore, at the center of the planar portion of the collimator 18 (the side opposite to the aforementioned top), a convex lens surface protruding outwards (or it may be a concave lens surface recessed inwards) may be formed. Additionally, the parabolic surface forming the conical outer peripheral surface of the collimator 18 is set within an angle range that allows light emitted from the LED in the peripheral direction to undergo total internal reflection within it, or it forms a reflective surface.

[0295] In addition, the LEDs are respectively disposed at predetermined positions on the surface of its circuit board, i.e., the LED substrate 102. The LED substrate 102 is configured and fixed relative to the collimator 18 such that the LEDs on its surface are respectively located at the center of the top of the convex conical shape (or the recess if the top has a recess).

[0296] According to this structure, under the action of the collimator 18, the light emitted from the LED, especially the light emitted from its central portion, is converged into parallel light by the convex lens surface forming the shape of the collimator 18. Furthermore, light emitted from other portions towards the periphery is reflected by the parabolic surface of the conical outer peripheral surface of the collimator 18, and similarly converged into parallel light. In other words, by using the collimator 18, which has a convex lens in its central portion and a parabolic surface in its peripheral portion, almost all the light generated by the LED can be output as parallel light, thereby improving the utilization efficiency of the generated light.

[0297] and then, Figure 9The light, transformed into approximately parallel light by collimator 18, is reflected by reflective light guide 304. A portion of this light, with a specific polarization, passes through reflective polarizer 49, and another portion, reflected by reflective polarizer 49, passes through light guide 304 again. This light is reflected by reflective plate 271, located opposite to the liquid crystal display panel 11 relative to reflective light guide 304. Here, the light undergoes polarization transformation twice by passing through λ / 4 waveplate 270, which acts as a phase difference plate. The light reflected by reflective plate 271 passes through light guide 304 again and is incident on reflective polarizer 49, which is positioned on the opposite side. Because the incident light has been polarized, it can pass through reflective polarizer 49, resulting in a uniform polarization direction incident on the liquid crystal display panel 11. As a result, all light from the light source can be utilized, achieving a 2-fold increase in geometrical optics efficiency. Furthermore, the polarization degree (extinction ratio) of the reflective polarizer is multiplied by the overall extinction ratio of the system, thus significantly improving the overall contrast of the display device by using the light source device of this embodiment. Additionally, by adjusting the surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflective plate 271, the reflection diffusion angle of light on each reflective surface can be adjusted. To improve the uniformity of light incident on the liquid crystal display panel 11, the surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflective plate 271 can be adjusted according to each design.

[0298] in addition, Figure 9 For polarized light incident perpendicular to the λ / 4 waveplate 270, the phase difference does not need to be λ / 4. Figure 9 In this structure, any phase difference plate that allows polarized light to pass through twice, thus changing its phase by 90° (λ / 2), is sufficient. The thickness of the phase difference plate can be adjusted accordingly based on the incident angle distribution of the polarized light.

[0299] <Example 4 of a display device>

[0300] Furthermore, using Figure 10 Another example illustrating the structure of an optical system such as a light source device in a display device (Example 4 of the display device). This is a structural example in the light source device of Example 3 of the display device, where a diffuser is used instead of the reflective light guide 304. Specifically, on the light emitting side of the collimator 18, two optical sheets (optical sheet 207A and optical sheet 207B) are used to transform the diffusion characteristics in the vertical and horizontal directions (front and back directions of the figure, not shown), so that light from the collimator 18 is incident between the two optical sheets (diffusers).

[0301] Alternatively, the aforementioned optical sheet can be replaced with a single sheet instead of a two-sheet structure. With a single-sheet structure, the vertical and horizontal diffusion characteristics are adjusted by the fine shapes of the front and back sides of the single optical sheet. Alternatively, multiple diffuser sheets can be used to share the effect. Here, in Figure 10 In the example, for the reflection diffusion characteristics determined by the front and back shapes of optical sheets 207A and 207B, the number of LEDs, the divergence angle of the LED substrate (optical element) 102, and the optical specifications of the collimator 18 can be used as design parameters for optimization, so that the surface density of the light beam emitted from the liquid crystal display panel 11 becomes uniform. That is, the diffusion characteristics are adjusted by using the surface shapes of multiple diffusers instead of a light guide.

[0302] Figure 10 In this example, the polarization transformation is performed using the same method as in Example 3 of the aforementioned display device. That is, in Figure 10 In this example, the reflective polarizer 49 can be configured to reflect S-polarized light (transmit P-polarized light). In this case, P-polarized light emitted from the light source, i.e., the LED, is transmitted, and the transmitted light enters the liquid crystal display panel 11. S-polarized light emitted from the light source, i.e., the LED, is reflected, and the reflected light passes through... Figure 10 The phase retardation plate 270 is shown. Light passing through the phase retardation plate 270 is reflected by the reflector plate 271. The light reflected by the reflector plate 271 passes through the phase retardation plate 270 again and is converted into P-polarized light. The polarized light passes through the reflective polarizer 49 and is incident on the liquid crystal display panel 11.

[0303] in addition, Figure 10 For polarized light incident perpendicular to the λ / 4 waveplate 270, the phase difference does not need to be λ / 4. Figure 10 In this structure, any phase retardation plate that allows polarized light to pass through twice, thus changing its phase by 90° (λ / 2), is sufficient. The thickness of the phase retardation plate can be adjusted accordingly based on the incident angle distribution of the polarized light. Furthermore, Figure 10 Similarly, in polarization transformation, the polarization design can be reversed compared to the above description (swapping the S polarization and P polarization).

[0304] Regarding the emitted light from the liquid crystal display panel 11, in typical TV-use devices, in the horizontal direction of the screen (using... Figure 12 (a) represented by the X-axis) and the vertical direction of the image (using) Figure 12 (b) representing the Y-axis all exhibit the same diffusion characteristics. In contrast, the diffusion characteristics of the emitted light beam from the liquid crystal display panel of this embodiment are as follows: Figure 12As shown in Example 1, the viewing angle is 13 degrees when the brightness is 50% of that of a front-viewing device (0 degrees), which is about 1 / 5 of the 62 degrees of a typical TV device. Similarly, for the vertical viewing angle, the reflection angle of the reflective light guide and the area of ​​the reflective surface are optimized to make it uneven vertically, suppressing the upper viewing angle to about 1 / 3 of the lower viewing angle. As a result, compared with existing LCD TVs, the amount of image light heading towards the viewing direction is significantly increased, with brightness more than 50 times.

[0305] Furthermore, if adopted Figure 12 The viewing angle characteristics shown in Example 2 indicate that the viewing angle is 5 degrees when the brightness is 50% of that of a front-viewing device (0 degrees), which is 1 / 12 of the 62 degrees of a typical TV device. Similarly, for the vertical viewing angle, the reflection angle of the reflective light guide and the area of ​​the reflective surface are optimized to make it uniform vertically and to suppress the viewing angle to about 1 / 12 of that of a typical TV device. As a result, the amount of image light heading towards the viewing direction is significantly increased compared to existing LCD TVs, with brightness more than 100 times greater.

[0306] By narrowing the viewing angle as described above, the amount of light beam heading in the viewing direction can be concentrated, thus significantly improving light utilization efficiency. As a result, even when using a typical LCD panel for TV applications, a significant increase in brightness can be achieved with the same power consumption by controlling the light diffusion characteristics of the light source device, enabling an image display device compatible with information display systems facing bright outdoor environments.

[0307] When using a large LCD panel, the light from the periphery of the screen is directed inwards so that it reaches the viewer when the viewer is facing the center of the screen, thereby improving the overall brightness of the screen. Figure 11 The convergence angles of the long and short sides of the panel were determined using the distance L between the viewer and the panel, and the panel size (16:10 aspect ratio) as parameters. In portrait mode, the convergence angle can be set accordingly for the short side. For example, with a 22″ panel used in portrait mode at a viewing distance of 0.8m, setting the convergence angle to 10 degrees will allow image light from the four corners of the screen to effectively reach the viewer.

[0308] Similarly, when viewing a 15″ panel in portrait mode at a viewing distance of 0.8m, setting the convergence angle to 7 degrees allows image light from the four corners of the screen to effectively reach the viewer. As described above, by directing image light from the periphery of the screen to the viewer in the most suitable position for viewing the center of the screen, based on the size of the LCD panel and whether it is used in portrait or landscape mode, the overall brightness of the screen can be improved.

[0309] As the basic structure, as described above Figure 9 The light source device is used to direct a beam of light with narrow-angle pointing characteristics onto the liquid crystal display panel 11. The brightness is modulated accordingly according to the image signal, so that the image information displayed on the screen of the liquid crystal display panel 11 is reflected on the retroreflector to obtain a spatial levitation image, which is then displayed outdoors or indoors via the transparent component 100.

[0310] Using the display device and light source device of one embodiment of the present invention described above, a spatial levitation image display device with higher light utilization efficiency can be realized.

[0311] <Examples of image display processing in spatial levitation image display devices>

[0312] Next, for one example of the technical problem to be solved by the image processing in this embodiment, using... Figure 13A The following explanation is provided. In the spatial levitation image display device 1000, from the user's perspective, the deep side of the spatial levitation image 3 is located inside the housing of the spatial levitation image display device 1000, and in sufficiently dark conditions, the user sees the background of the spatial levitation image 3 as black.

[0313] Here, use Figure 13A This section explains an example of displaying the character "Panda" 1525 within a spatial levitation image 3. First, Figure 3 The image control unit 1160 is for such Figure 13A The image shown in (1) includes a pixel region depicting the character “panda” 1525 and a transparent information region 1520 serving as a background image, and the pixel region depicting the character “panda” 1525 and the transparent information region 1520 serving as a background image are distinguished.

[0314] Regarding the method for distinguishing and recognizing character images from background images, for example, the image processing of the image control unit 1160 can be configured to process the background image layer and the character image layer located in front of the background image layer as different layers, and to distinguish and recognize the character image and the background image based on the overlap relationship when these layers are composited.

[0315] Here, the image control unit 1160 identifies the black pixels of the pixels depicting objects such as character images as different from the transparent information pixels. However, the pixel brightness of both the black pixels depicting objects and the transparent information pixels is 0. In this case, when displaying the spatial levitation image 3, the black pixels in the image depicting the character "panda" 1525 have no brightness difference from the pixels in the transparent information area 1520, which serves as the background image. Therefore, in the spatial levitation image 3, as... Figure 13AAs shown in (2), the black pixels in the image depicting the character "Panda" 1525 and the pixels in the transparent information area 1520 have no brightness and are perceived by the user as optically identical black spaces. That is, the black parts in the image depicting the object, namely the character "Panda" 1525, blend into the background, and only the non-black parts in the character "Panda" 1525 can be recognized as images floating in the display area of ​​the spatial floating image 3.

[0316] For one example of image processing in this embodiment, using Figure 13B Please provide an explanation. Figure 13B This indicates that it can better solve the problem. Figure 13A This image illustrates an example of image processing addressing the technical problem of black areas of the object blending seamlessly into the background. Figure 13B In (1) and (2), the display state of the spatial levitation image 3 is shown on the upper side, and the input / output characteristics of the image processing of the object's image are shown on the lower side. Additionally, the image of the object (character "Panda" 1525) and / or its corresponding data can be obtained from... Figure 3 It can be read from the storage unit 1170 or the memory 1109. Alternatively, it can be input from the image signal input unit 1131. Alternatively, it can be obtained via the communication unit 1132.

[0317] Here, in Figure 13B In state (1), the input-output characteristics of the object's image processing are linear without any special adjustment. In this case, it is related to... Figure 13A In (2) the same display state, the black image area of ​​the object blends into the background. In contrast, in Figure 13B In (2), the image control unit 1160 of this embodiment adjusts the input-output characteristics of the image processing of the object (character "panda" 1525) to the input-output characteristics shown below.

[0318] That is, the image control unit 1160 performs image processing on the image of the object (character "Panda" 1525) with the following input-output characteristics: for each pixel of the input image, the brightness value of the pixel in the low-brightness area is increased to obtain an output pixel. After the image processing with these input-output characteristics is performed on the image of the object (character "Panda" 1525), the image including the image of the object (character "Panda" 1525) is input to the display device 1 for display. Thus, the display state of the spatial levitation image 3 is as follows: Figure 13BAs shown in the upper part of (2), the brightness of the black pixel area in the image depicting the character "Panda" 1525 is increased. As a result, the black areas in the image depicting the character "Panda" 1525 are not blended into the black background, allowing users to distinguish them and better display the object.

[0319] That is, by using Figure 13B The image processing of (2) displays an area of ​​the image of the object, namely the character "panda" 1525, which can be distinguished from the black background inside the housing of the spatial levitation image display device 1000 seen through the window, thus improving the recognizability of the object. Therefore, even if the object was before the aforementioned image processing (i.e., from...) Figure 3 When the image of the object and / or its corresponding data is read from the storage unit 1170 or the memory 1109, or when the image of the object is input from the image signal input unit 1131, or when the data of the object is obtained via the communication unit 1132, etc., an object whose pixels constituting the object include pixels with a brightness value of 0 can also be transformed into an object with increased brightness value of pixels in a low brightness area through image processing of the input-output characteristics performed by the image control unit 1160, and then displayed on the display device 1, and transformed into a spatial levitation image 3 by the optical system of the spatial levitation image display device 1000.

[0320] That is, the pixels of the object after image processing that constitutes the input and output characteristics do not contain pixels with a brightness value of 0. After being transformed into such a state, it is displayed on the display device 1 and transformed into a spatial levitation image 3 by the optical system of the spatial levitation image display device 1000.

[0321] In addition, Figure 13B In the image processing of (2), it is implemented only on a region of the image of the object (character "Panda" 1525). Figure 13B The image processing method with input-output characteristics of (2) is configured, for example, in the image processing of the image control unit 1160, to process the background image layer and the character image layer located in front of the background image layer as different layers, and to perform the processing on the character image layer. Figure 13B Image processing of the input and output characteristics of (2) does not apply to the background image layer.

[0322] These layers are then composited, resulting in, as shown Figure 13B As shown in (2), only the character image is subjected to image processing that brightens the low-brightness areas of the input image. Alternatively, as another method, it can be configured such that after compositing the character image layer with the background image layer, only the area of ​​the character image is processed. Figure 13B Image processing with input-output characteristics of (2).

[0323] Furthermore, in image processing that brightens low-brightness areas in the input-output characteristics of an input image, the input-output image characteristics used are not limited to... Figure 13B Example (2). Any image processing that can brighten low brightness is acceptable, such as so-called brightness adjustment. Alternatively, image processing as disclosed in International Publication No. 2014 / 162533 can be performed to improve recognizability by controlling the gain used to change the weights of the Retinex processing.

[0324] Based on the above explanation Figure 13B (2) Image processing can make black areas in the area of ​​an image that depicts characters and objects not blend into the black background, so that users can recognize them and achieve better display.

[0325] In addition, Figure 13A , Figure 13B In the example, a spatially suspended image display device with a seemingly black background (e.g.) Figures 4A to 4G Spatial levitation image display device 1000 Figure 4I , Figure 4J Taking a spatial levitation image display device 1000 (with the rear side window shaded) as an example, its technical problems and improved image processing have been explained. However, this image processing is also effective in devices other than these spatial levitation image display devices.

[0326] Specifically, in Figure 4H Space levitation image display device 1000 and Figure 4I , Figure 4J In the spatial levitation image display device 1000 with the rear side window unobstructed, the background of the spatial levitation image 3 is not black, but rather the scenery behind the spatial levitation image display device 1000 through the window. In this case, Figure 13A and Figure 13B The technical issues described also exist.

[0327] That is, the black parts of the image depicting the object, namely the character "panda" 1525, blend into the scenery behind the space-suspended image display device 1000, which is separated by a window. The same applies in this case, through the use of... Figure 13B The image processing of (2) enables the black parts in the image depicting the object, namely the character "panda" 1525, to be distinguishably identified from the scenery behind the space-suspended image display device 1000 through the window, thereby improving the recognizability of the object.

[0328] That is, by using Figure 13BThe image processing of (2) shows that the area of ​​the image of the object, namely the character "panda" 1525, can be distinguished from the scenery behind the space-suspended image display device 1000 through the window, and the object, namely the character "panda" 1525, is better identified in front of the scenery, thus improving the recognizability of the object.

[0329] In addition, Figure 4K , Figure 4L , Figure 4M In the spatial levitation image display device 1000, as described above, when other images (such as images from the transmissive self-emissive image display device 1650 or the second display device 1680) are displayed at a depth position different from that of the spatial levitation image 3, the background of the spatial levitation image 3 is not black but rather that other image. In this case, Figure 13A and Figure 13B The technical issues described also exist.

[0330] That is, the black portion of the image depicting the object, namely the character "panda" 1525, blends seamlessly with the other images shown at a depth position different from that of the spatially suspended image 3. The same applies in this case, through the use of... Figure 13B The image processing of (2) enables the black parts in the image depicting the object, namely the character “panda” 1525, to be distinguishably identified from the other images mentioned above, thus improving the recognizability of the object.

[0331] That is, by using Figure 13B The image processing of (2) shows that the area of ​​the image of the object, namely the character "panda" 1525, can be distinguished from the other images mentioned above, and the object, namely the character "panda" 1525, can be better identified as being in front of the other images mentioned above, thus improving the recognizability of the object.

[0332] For one example of image display processing in this embodiment, using Figure 13C Please provide an explanation. Figure 13C This is an example of image display in this embodiment, showing a display of both the spatially suspended image 3 and other images, namely the second image 2050. The second image 2050 can correspond to... Figure 4K or Figure 4L The image displayed by the transmissive self-emissive image display device 1650. Additionally, the second image 2050 can also correspond to... Figure 4M The second display device 1680 displays images.

[0333] Right now, Figure 13C The image shows an example of Figure 4K , Figure 4L , Figure 4MThis is a specific example of an image display example of a spatial levitation image display device 1000. In this example, a bear character is displayed in spatial levitation image 3. The area outside the bear character in spatial levitation image 3 is black, as spatial levitation images are transparent. Additionally, the second image 2050 is a background image depicting a plain, mountains, and the sun.

[0334] Here, Figure 13C In the image, the spatial levitation image 3 and the second image 2050 are displayed at different depth positions. The user 230 views the two images, spatial levitation image 3 and the second image 2050, in the line of sight of arrow 2040, thus allowing the user 230 to view the images in an overlapping state. Specifically, the bear character in spatial levitation image 3 can be seen overlapping in front of the background of plains, mountains, and sun depicted in the second image 2050.

[0335] Here, the spatially suspended image 3 is projected as a real image in the air, so when the user 230 slightly moves their viewpoint, they can discern the distance between the spatially suspended image 3 and the second image 2050 due to parallax. Therefore, the user 230 can view the two images in an overlapping state, while simultaneously experiencing a stronger sense of spatial suspension from the spatially suspended image 3.

[0336] For one example of image display processing in this embodiment, using Figure 13D Please provide an explanation. Figure 13D (1) is Figure 13C In this example of image display, the image of the spatial levitation image 3 is viewed from the direction of the user 230's gaze. Here, a bear character is displayed in the spatial levitation image 3. The area outside the bear character in the spatial levitation image 3 is black, as the spatial levitation image is transparent.

[0337] Figure 13D (2) is Figure 13C In this example of image display, the second image 2050 is viewed from the direction of the user 230's line of sight. In this example, the second image 2050 is a background image depicting a plain, mountains, and the sun.

[0338] Figure 13D (3) represents Figure 13C In the example of image display in this embodiment, a diagram showing the state of the second image 2050 and the spatial levitation image 3 being seen overlaid in the direction of the user 230's gaze. Specifically, the bear character of the spatial levitation image 3 can be seen overlaid in front of the background of the plains, mountains, and sun depicted in the second image 2050.

[0339] Here, when both the spatial levitation image 3 and the second image 2050 are displayed simultaneously, it is preferable to maintain a balance in the brightness of both images to better ensure the recognizability of the spatial levitation image 3. If the second image 2050 is too bright compared to the brightness of the spatial levitation image 3, the displayed image of the spatial levitation image 3 will become transparent, allowing the background, i.e., the second image 2050, to be seen clearly and transparently.

[0340] Therefore, the output of the light source of the spatial levitation image 3 and the display image brightness of the display device 1, the output of the light source of the display device displaying the second image 2050 and the display image brightness of the display device can be set such that the brightness per unit area of ​​the spatial levitation image 3 at the display position of the spatial levitation image 3 is greater than the brightness per unit area of ​​the image light from the second image 2050 to the display position of the spatial levitation image 3.

[0341] Furthermore, as long as this condition is met when both the spatial floating image 3 and the second image 2050 are displayed simultaneously, control can be implemented to reduce the brightness of the second image 2050 by decreasing the output of the light source of the display device displaying the second image 2050 and / or the brightness of the displayed image on the display device. This control can be achieved through... Figure 3 The control unit 1110 controls the display device 1 and the display device that displays the second image 2050. Figure 4K or Figure 4L Transmissive self-emissive image display device 1650 or Figure 4M This is achieved through a second display device 1680.

[0342] Furthermore, during the switching from the first display mode to the second display mode, if it is necessary to reduce the brightness of the second image 2050, the brightness can be reduced uniformly across the entire screen of the second image 2050. Alternatively, instead of uniformly reducing the brightness across the entire screen of the second image 2050, the portion of the spatial levitation image 3 where the object to be displayed is made to have the highest brightness reduction effect, with the brightness reduction effect gradually reduced around its perimeter. This is because reducing the brightness of the second image 2050 only for the overlapping portion of the spatial levitation image 3 within the second image 2050 is sufficient to ensure the recognizability of the spatial levitation image 3.

[0343] Here, the spatial levitation image 3 and the second image 2050 are displayed at different depth positions, so when the user 230 slightly changes their viewpoint, the overlapping position of the spatial levitation image 3 on the second image 2050 will change due to parallax. Therefore, in the switching from the first display mode to the second display mode, when the brightness of the entire screen of the second image 2050 is reduced unevenly, it is not preferable to sharply reduce the brightness based on the outline of the object displayed in the spatial levitation image 3. Instead, it is preferable to gradually change the brightness reduction effect according to the position as described above, and perform a gradual processing of the brightness reduction effect.

[0344] Furthermore, if the position of the object displayed in the spatial levitation image 3 is approximately the center of the spatial levitation image 3, then in the spatial levitation image display device 1000, the position with the highest brightness reduction effect in the gradient processing of the brightness reduction effect can be set to the center of the spatial levitation image 3.

[0345] Based on the image display processing of this embodiment described above, user 230 is able to better view the spatial levitation image 3 and the second image 2050.

[0346] Alternatively, it can be controlled so that the second image 2050 is not displayed when the spatial levitation image 3 is displayed. When the second image 2050 is not displayed, the spatial levitation image 3 is more recognizable, so it is suitable for applications such as the spatial levitation image display device 1000 where the user must reliably see the spatial levitation image 3 when it is displayed.

[0347] <Example 2>

[0348] As Embodiment 2 of the present invention, another example of the structure of the spatial levitation image display device will be described. Furthermore, the optical system incorporated in the spatial levitation image display device described in Embodiment 1 is modified in this embodiment. Figure 14 (1) or Figure 14 The optical system shown in (2) is described in this embodiment. The differences from Embodiment 1 are explained in this embodiment, and repeated descriptions of the same structure as in Embodiment 1 are omitted. In addition, in the following description of this embodiment, it is specified that polarization (polarized light) and another polarization (polarized light) are polarizations (polarized light) that are 90° out of phase.

[0349] Figure 14 (1) is an example of the optical system and optical path of this embodiment. Figure 14 The optical system shown in (1) is in Figure 2C In the optical system, the display device 1 is moved closer to the polarization separation component 101B, making the overall optical system more compact. For Figure 14 (1) is marked with Figure 2CThe same reference numerals are used in the accompanying drawings, and repeated detailed descriptions are omitted.

[0350] Figure 14 In (1), with Figure 2C Similarly, image light with a predetermined polarization (P-polarization in the figure) emitted from display device 1 travels vertically from the image display surface of display device 1. Here, the polarization separation component 101B and... Figure 2C Similarly, a predetermined polarized light (P-polarized light in the figure) emitted from the display device 1 is selectively transmitted, while another polarized light (S-polarized light in the figure) is reflected.

[0351] Thus, the image light of a predetermined polarization (P-polarized in the figure) traveling vertically from the image display surface of the display device 1 passes through the polarization separating member 101B and reaches the retroreflector 2 to which the λ / 4 waveplate 21 is attached. The image light, retroreflected on the retroreflector 2 and traveling again towards the polarization separating member 101B, passes through the λ / 4 waveplate 21 twice, thereby changing from the predetermined polarization (P-polarized in the figure) emitted from the display device 1 to another polarization (S-polarized in the figure). The image light traveling again towards the polarization separating member 101B is now of another polarization (S-polarized in the figure), and is therefore reflected on the polarization separating member 101B towards the position where the user should be located. The direction of travel of the image reflected on the polarization separating member 101B is determined based on the configuration angle of the polarization separating member 101B.

[0352] Figure 14 In example (1), the image light traveling toward the polarization separation component 101B is reflected at a right angle onto the polarization separation component 101B and travels as shown in the figure. The image light reflected onto the polarization separation component 101B forms a spatial levitation image 3A. The spatial levitation image 3A can be viewed well by the user from the direction of arrow A.

[0353] Here, regarding the retroreflective characteristics of the retroreflector 2, there exists a relationship where the optical path length of the image light emitted from the display device 1 to the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 to the formation position of the spatially suspended image 3A. The formation position of the spatially suspended image 3A in the direction of travel of the image light reflected from the polarization separation component 101B is determined by this relationship.

[0354] Figure 14 In example (1), the display device 1, the polarization separation component 101B, and the retroreflector 2 are made to... Figure 2C The example is configured more closely. This allows for a more compact overall optical system design. However, the space-suspended image 3A... Figure 14The amount by which the optical system of (1) floats is not large. For example, as an indicator of the amount by which the spatial levitation image 3A floats from the optical system, the figure shows the distance from the position where the light from the center of the image light is reflected on the polarization separation component 101B to the position where the spatial levitation image 3A is formed by the image light ( Figure 14 In example (1), it is L1).

[0355] In addition, regarding Figure 14 In the polarization design of the optical system of (1), the characteristics of P polarization and S polarization can also be interchanged. Specifically, the specified polarization of the image light emitted from the display device 1 can be made S polarization, and the characteristics of P polarization and S polarization can be interchanged regarding the reflection characteristics of the polarization separation component 101B. In this case, the P polarization and S polarization shown in the figure become opposite, but the optical design of the optical path, etc., can be implemented exactly the same.

[0356] Next, in Figure 14 Another example of the optical system and optical path of this embodiment is shown in (2). Figure 14 The optical system of (2) is designed to achieve [the desired result]. Figure 14 The optical system of (1) has the same compactness, while increasing the amount of space-suspended images floating from the optical system, and thus changing Figure 14 The structure of the optical system in (1). For Figure 14 (2) is marked with Figure 14 (1) The same reference numerals are used in the same figures, and repeated detailed descriptions are omitted.

[0357] Figure 14 In (2), with Figure 14 Similarly, (1) the image light of a predetermined polarization (P-polarization in the figure) emitted from the display device 1 travels vertically from the image display surface of the display device 1. Here, the polarization characteristics of the polarization separation component 101B are configured in accordance with... Figure 14 The phase difference between (1) and (2) is 90 degrees. Image light of a specified polarization (P polarization in the figure) traveling vertically from the image display surface of the display device 1 passes through the polarization separation member 101B.

[0358] Here, with Figure 14 Unlike (1), in front of the image light passing through the polarization separation component 101B, instead of a retroreflector 2 with a λ / 4 wave plate 21 attached, a specular reflector 4 with a λ / 4 wave plate 21B attached is arranged. Here, the reflection on the specular reflector 4 is specular reflection (also called orthographic reflection), not retroreflection.

[0359] Therefore, the image light passing through the polarization separation component 101B undergoes specular reflection on the mirror reflector 4 to which the λ / 4 waveplate 21B is attached. The image light, after being specularly reflected on the mirror reflector 4 and traveling again towards the polarization separation component 101B, passes through the λ / 4 waveplate 21 twice, thus changing from the predetermined polarized light (P-polarized light in the figure) emitted from the display device 1 to another polarized light (S-polarized light in the figure). The image light traveling again towards the polarization separation component 101B is now another polarized light (S-polarized light in the figure), and therefore reflects off the polarization separation component 101B.

[0360] Here, Figure 14 The orientation of the polarization separation component 101B in (2) is related to... Figure 14 Because (1) is different, the image light reflected from the polarization separation component 101B travels in the opposite direction to the position where the user should be located. A retroreflector 2 with a λ / 4 waveplate 21C attached is positioned in front of the image light reflected from the polarization separation component 101B. The image light is retroreflected by the retroreflector 2. The image light, retroreflected by the retroreflector 2 and traveling again towards the polarization separation component 101B, passes through the λ / 4 waveplate 21C twice, thereby changing from another polarized light (S-polarized light in the figure) back to the specified polarized light (P-polarized light in the figure).

[0361] The image light that travels again towards the polarization separation component 101B is predefined polarized light (P-polarized light in the figure), so it travels through the polarization separation component 101B to the position where the user should be. The image light passing through the polarization separation component 101B forms a spatial levitation image 3B. The spatial levitation image 3B can be viewed well by the user from the direction of arrow A.

[0362] Here, Figure 14 In (2), it is also related to Figure 14 Similarly, regarding the retroreflective characteristics of the retroreflector 2, there exists a relationship in which the optical path length of the image light emitted from the display device 1 to the retroreflector 2 is equal to the optical path length of the image light emitted from the retroreflector 2 to the formation position of the spatially suspended image 3B. The formation position of the spatially suspended image 3B in the direction of travel of the image light passing through the polarization separation component 101B is determined by this relationship.

[0363] Figure 14 The optical path length of the image light emitted from the display device 1 to the retroreflector 2 in (2) is related to... Figure 14 The optical path length of the image light emitted from the display device 1 to the retroreflector 2 in (1) is longer. This is because... Figure 14 In the optical system of (2), the optical path length for the image light emitted from the display device 1 to reach the retroreflector 2 is increased by Figure 14The optical path that does not exist in the optical system of (1) is the reciprocating optical path between the polarization separation component 101B and the mirror reflector 4.

[0364] thus, Figure 14 In the optical system of (2), the distance from the position of the light ray passing through the polarization separation component 101B from the center part of the image light to the position of the spatially suspended image 3B formed by the image light is ( Figure 14 In example (2), it is L2), and Figure 14 In the optical system of (1), the distance from the position where the light ray from the center of the image light is reflected on the polarization separation component 101B to the position where the spatially suspended image 3A is formed by the image light is ( Figure 14 In example (1), L1 is much longer.

[0365] In addition, regarding Figure 14 In the polarization design of the optical system of (2), the characteristics of P polarization and S polarization can also be interchanged. Specifically, the specified polarization of the image light emitted from the display device 1 can be made S polarization, and the characteristics of P polarization and S polarization can be interchanged regarding the reflection characteristics of the polarization separation component 101B. In this case, the P polarization and S polarization shown in the figure become opposite, but the optical design of the optical path, etc., can be achieved exactly the same.

[0366] According to the above description of Embodiment 2 of the present invention Figure 14 (1) and Figure 14 The optical system of (2) enables a more compact optical system. In particular, according to... Figure 14 The optical system of (2) can make the amount of space-suspended images float up from the optical system by a larger amount with a more compact optical system.

[0367] In addition, in order to Figure 14 (1) or Figure 14 When the optical system of (2) is installed in the space levitation image display device, simply replace the optical system in the space levitation image display device described in Example 1 with Figure 14 (1) or Figure 14 The optical system of (2) can achieve this. Specifically, it can be... Figure 14 The optical system of (1) and Figure 4E , Figure 4F , Figure 4G , Figure 4H , Figure 4I , Figure 4J , Figure 4K , Figure 4L or Figure 4M The optical system of the spatial levitation image display device is replaced. In this case, because the optical system becomes more compact, the housing of the spatial levitation image display device in each figure can be made smaller.

[0368] In addition, specifically, it can be Figure 14 The optical system of (2) and Figure 4E , Figure 4F , Figure 4G , Figure 4K or Figure 4L The optical system of the levitation image display device is replaced. In this case, the amount by which the levitation image rises from the optical system is greater. In addition, since the optical system becomes more compact, the housing of the levitation image display device in each figure can be made smaller.

[0369] <Example 3>

[0370] As Embodiment 3 of the present invention, a structural example of a spatial levitation image display device will be described. The spatial levitation image display device of Embodiment 3 can also utilize the structures of Embodiments 1 and 2 as its basic structure. In the spatial levitation image display device of Embodiment 3, the structure of the optical system can, for example, utilize... Figure 2A Based on this, the shell, or in other words, the main body structure, can be used for example... Figure 4A As a foundation, Embodiment 3 is a system (sometimes described as a spatial levitation image display system, etc.) having a spatial levitation image display device and external devices connected thereto. External devices are, for example, portable terminals such as smartphones, tablets, and wearable devices (smartwatches, etc.) held by the user. Portable terminals can also be referred to as mobile information processing terminal devices.

[0371] The basic functions implemented in the spatial levitation image display device / system of Example 3 are described below. This spatial levitation image display system connects and collaborates with a user's portable terminal via communication. The system is capable of displaying images / images owned by the user, such as images displayed on the user's screen (screen) within the portable terminal (sometimes referred to as target images, etc.), as a spatial levitation image 3. For this purpose, the user must perform an operation / action to bring the portable terminal into contact with the spatial levitation image 3 (and its corresponding predetermined imaging position and display range) of the spatial levitation image display device. This operation / action is a predetermined operation / action, sometimes referred to as an insertion operation / contact operation, etc. The spatial levitation image display device uses sensors, etc., to determine / detect this contact operation. In this example, for instance, while the portable terminal sends a display request based on the user's input operation, the user performs a contact operation by inserting / contacting the portable terminal to a planar position of the spatial levitation image 3.

[0372] The levitation image display device or portable terminal generates a display request / display instruction based on this contact operation to display the target image on one side of the portable terminal as the levitation image 3. In other words, the levitation image display device / portable terminal generates this display request / display instruction when it determines that a predetermined contact operation has occurred (that the predetermined contact operation satisfies the predetermined conditions regarding contact between the portable terminal and the levitation image 3). The levitation image display device, in particular... Figure 3 The image control unit 1160 receives the display request and receives and obtains the target image data from the portable terminal via communication. Then, the spatial levitation image display device performs image processing on the display device 1 to display the target image as a spatial levitation image 3.

[0373] This function can also be as described below. The user's portable terminal sends the aforementioned display request along with the contact operation (in other words, before, during, or immediately after contact). Based on the contact operation, the spatial levitation image display device receives the display request from the portable terminal and authorizes / activates the request, performing the same process as described above to display the target image as spatial levitation image 3. This display request can be sent along with the target image data. This function differs in whether the display request is generated by the spatial levitation image display device or by the portable terminal, but regardless of which is generated, the point is the same: automatically displaying the target image as spatial levitation image 3 based on a specified contact operation.

[0374] This function can also be as described below. Initially, the portable terminal does not send a display request, and the user brings the portable terminal into contact with the spatial levitation image 3. Upon determining / detecting the specified contact operation, the spatial levitation image display device sends a confirmation message to the portable terminal via communication, indicating acceptance of the display request, or in other words, that the image can be displayed using this function. Upon receiving this confirmation message, the portable terminal sends the target image data to the spatial levitation image display device. The target image data may include a display request. Upon receiving the target image data, the spatial levitation image display device displays the target image as the spatial levitation image 3.

[0375] [Spatial Suspended Image Display System]

[0376] Figure 15 This illustrates the structure of the spatial levitation image display system, or system 3000, as described in Embodiment 3. System 3000 is a system formed by connecting the spatial levitation image display device 1000 and the portable terminal 2000 of user 230 via a communication network such as LAN 3020 and / or Internet 3010. Figure 15 The diagram schematically illustrates the housing 1190 (in other words, the main body) of the space levitation image display device 1000, the space levitation image 3, and the communication unit 1132. Figure 3 Part of the housing 1190. As described above, it is installed within the housing 1190. Figure 3 The constituent elements are shown. Additionally... Figure 3 The control unit 1110 and communication unit 1132 can be installed inside or outside the housing 1190. Outside the housing 1190, a camera of the camera unit 1180, an air operation detection sensor 1351, a microphone, and a speaker can be installed.

[0377] This system 3000 enables communication between the portable terminal 2000 and the spatial levitation image display device 1000 via any communication interface. The portable terminal 2000 and the spatial levitation image display device 1000 can communicate directly, for example, using short-range wireless communication, or they can communicate via the Internet 3010 or other means. Furthermore, because this system 3000 needs to transmit image data between the portable terminal 2000 and the spatial levitation image display device 1000, a certain level of communication speed is required. Therefore, Figure 15 In this example, the portable terminal 2000 and the spatial levitation image display device 1000 are connected via LAN 3020. That is, the portable terminal 2000 and the spatial levitation image display device 1000 are connected to the same network via a router or the like, and are configured to recognize each other using IP addresses, etc. However, they are not limited to this; they can also communicate using methods such as mirroring.

[0378] The portable terminal 2000 includes an application 2010 and an image 2020. The application 2010 is, for example, any application that displays the image 2020 on the screen of the portable terminal 2000's display. The application 2010 can be a general operating system or application, or a dedicated application for collaboration, communication, etc., with the spatial levitation image display device 1000 of the system 3000 of Embodiment 3. Data from the application 2010 can be distributed from the spatial levitation image display device 1000 or from a server on the Internet 3010. The image 2020 is image data serving as a target image. The image 2020 can be stored on a server on the Internet 3010 or distributed from the spatial levitation image display device 1000. The image 2020 can be a still image or a moving image. The image 2020 can also be data in a program or other format.

[0379] The application 2010 of the portable terminal 2000 can generate a display request 2030 for displaying the target image (image 2020) as a spatial levitation image 3 and send it to the spatial levitation image display device 1000.

[0380] Figure 16Here is an outline of the functions of the spatial levitation image display system of Embodiment 3. (1). First, the user 230 has a target image 1601, etc. in his own portable terminal 2000. The target image 1601 is an image that is displayed as the target of the spatial levitation image 3.

[0381] (2). User 230 performs contact operation 1600, causing portable terminal 2000 to be inserted into the display range 3R of spatial levitation image 3 of spatial levitation image display device 1000 and come into contact with it. At this time, portable terminal 2000 can send display request 2030. Display request 2030 is a request or instruction to display target image 1601 as spatial levitation image 3.

[0382] (3) When the spatial levitation image display device 1000 detects the touch operation 1600, it displays the target image 1601 of the portable terminal 2000 as the spatial levitation image 3, i.e., image 1602. At this time, the spatial levitation image display device 1000 can receive the display request 2030 and obtain the target image 1601 of the portable terminal 2000 based on the display request 2030. The spatial levitation image display device 1000 can send a certain response to the display request 2030 to the portable terminal 2000.

[0383] Figure 17 This illustrates a structural example of a portable terminal 2000. The portable terminal 2000, housed in a casing 2090, includes a control unit 20011, a display panel 20012, an external power input IF (interface) 20013, a power supply 20014, a secondary battery 20015, a storage unit 20016, an image control unit 20017, an attitude sensor 20018, an operation input unit 20019, a communication unit (including an antenna) 20020, a sound output unit (including a speaker) 20021, a sound input unit (including a microphone) 20022, an image signal input unit 20023, a sound signal input unit 20024, a camera unit 20025, a memory 20026, and a non-volatile memory 20027. These components are connected via a bus or similar architecture.

[0384] The control unit 20011 includes processors and other components that control the portable terminal 2000 as a whole and its individual parts. The control unit 20011 loads data such as programs stored in the non-volatile memory 20027 or the storage unit 20016 into the main memory 20026 and executes the processing according to the program. This enables various functions. The control unit 20011 can cooperate with the program stored in the main memory 20026 to perform calculations based on information obtained from various parts within the portable terminal 2000. Figure 15 The application 2010 is executed, for example, by the control unit 20011 or the image control unit 20017.

[0385] The memory 20026 stores image data for display on the display panel 20012, control data for the portable terminal 2000, etc. The control unit 20011 can read programs from various software such as the storage unit 20016, load them, and store them in the memory 20026. The non-volatile memory 20027 and the storage unit 20016 hold various data and information used in the portable terminal 2000. The data / information stored in the non-volatile memory 20027 and the storage unit 20016 includes, for example, data for various operations displayed on the display panel 20012, display icons, data of objects for user 230 to operate, layout information, etc.

[0386] The display panel 20012 is a display unit, such as a touch panel, which is both a display unit and an input unit that receives touch operation input. Images are displayed on the screen (image) of the display panel 20012. The display panel 20012 includes a touch sensor to receive touch operation input performed by the user 230's finger or similar device. The display panel 20012 can use liquid crystal panels, organic EL panels, etc.

[0387] Communication unit 20020 is a device equipped with various communication interfaces. In communication unit 20020, communication interfaces / communication methods include, for example, mobile communication interfaces such as 4G and 5G, wireless LAN communication interfaces such as Wi-Fi (registered trademark), and short-range communication interfaces such as Bluetooth (registered trademark) or NFC. The communication unit 20020 of portable terminal 2000 can use these communication methods to communicate with the communication unit 1132 of spatial levitation image display device 1000. Figure 3 Communication is established between the two devices. Similarly, from the perspective of the spatial levitation image display device 1000, the communication unit 1132 is equipped with these communication interfaces / communication methods. The communication unit 20020 can communicate with communication devices (such as wireless base stations) connected to the Internet 3010 using any communication method. Thus, the portable terminal 2000 can communicate with servers connected to the Internet 3010.

[0388] Power supply 20014 converts the AC current input from the external power input IF (interface) 20013 into DC current, supplying the required DC current to each part of the portable terminal 2000. Secondary battery 20015 stores the power (electrical power) supplied by power supply 20014. When no power is supplied from the external power input IF20013, secondary battery 20015 supplies power to the parts that require it.

[0389] The video signal input unit 20023 connects to an external video output device to input video data. The video signal input unit 20023 can utilize various digital video input interfaces. For example, it can use video input interfaces based on the HDMI (High-Definition Multimedia Interface) standard, the DVI (Digital Visual Interface) standard, or the DisplayPort standard. Alternatively, analog video input interfaces such as analog RGB and component video can also be provided. The video signal input unit 20023 can also be various USB interfaces, etc.

[0390] The audio signal input unit 20024 connects to an external audio output device to input audio data. The audio signal input unit 20024 can use an HDMI standard audio input interface, an optical digital terminal interface, or a coaxial digital terminal interface, etc. The audio signal input unit 20024 can also be various USB interfaces, etc. When using an HDMI standard interface, the video signal input unit 20023 and the audio signal input unit 20024 can be configured as an interface integrating the terminal and cable.

[0391] The audio output unit 20021 can output audio based on the audio data input to the audio signal input unit 20024 and the audio data stored in the storage unit 20016. The audio output unit 20021 can be configured as a speaker or may have a headphone jack. Additionally, the audio output unit 20021 can also output built-in operation tones and error warning tones. The audio output unit 20021 can also employ a structure that outputs digital signals to external devices, similar to the Audio Return Channel function specified in the HDMI standard.

[0392] The sound input unit 20022, including a microphone, collects sounds from the vicinity of the portable terminal 2000 and converts them into signals to generate sound signals. The sound input unit 20022 can use the microphone to record the voice of the user 230 or other people and generate sound signals. The control unit 20011 and others perform sound recognition processing based on the sound signals to obtain text information.

[0393] The camera unit 20025 is, for example, a camera with an image sensor. The portable terminal 2000 may have a camera (front-facing camera) mounted on the side of the display panel 20012 in the housing 2090, or it may have a camera (rear-facing camera) mounted on the opposite side, the back. In this embodiment, the camera unit 20025 has both a front-facing camera and a rear-facing camera.

[0394] Storage unit 20016 is a storage device for recording various data and information such as image data, audio data, and programs. Storage unit 20016 can be configured, for example, by a magnetic recording medium recording device such as a hard disk drive (HDD) or a semiconductor device memory such as a solid-state drive (SSD). In storage unit 20016, various data and information such as image data, audio data, and programs can be pre-recorded, for example, at the time of product shipment. Furthermore, storage unit 20016 can record various data and information obtained from external servers, etc., via communication unit 20020. Image data, etc., recorded in storage unit 20016 are processed by image control unit 20017 and output and displayed on display panel 20012. Portable terminal 2000 can output and send various data and information recorded in storage unit 20016 to external servers, etc., via communication unit 20020.

[0395] The image control unit 20017 performs various controls on the image signals to be input to the display panel 20012. The image control unit 20017 may also be referred to as an image processing circuit, image processing unit, or image processing unit. The image control unit 20017 can be constructed from hardware such as an ASIC, FPGA, or video processor. For example, the image control unit 20017 performs image switching control, switching which image signal from the image signal stored in the memory 20026 or the image signal (image data) input to the image signal input unit 20023 is input to the display panel 20012 for display. Furthermore, the image control unit 20017 can also control image processing of the image signals input from the image signal input unit 20023 and the image signals stored in the memory 20026. Image processing includes, for example, scaling processing (enlarging, reducing, and distorting the image), brightness adjustment processing (changing brightness), contrast adjustment processing (changing the contrast curve of the image), and Retinex processing (decomposing the image into light components and changing the weights of each component). The image control unit 20017 may include a video memory or the like to store image data to be input to the display panel 20012.

[0396] The attitude sensor 20018 is a sensor composed of a combination of a gyroscope sensor, a gravity sensor, an accelerometer, and a geomagnetic sensor, capable of detecting the attitude of the portable terminal 2000. Attitude can be expressed, for example, using orthogonal three-axis (X, Y, Z) directions and angles in space. The control unit 20011 can control the actions of various components based on the attitude detection results from the attitude sensor 20018. In addition, the portable terminal 2000 may also include a GPS receiver, non-contact sensors, an illumination sensor, a distance sensor, etc.

[0397] The operation input unit 20019 is a device for user 230 to perform operation input, such as a power button, volume button, etc.

[0398] [Image display method]

[0399] Figures 18A to 18D This is an explanatory diagram of the image display method implemented in system 3000 of embodiment 3 through the cooperation of spatial levitation image display device 1000 and portable terminal 2000. Figures 18A to 18D The image shows a scenario where a user 230 brings a portable terminal 2000 into contact with the display area 3R of the spatial levitation image 3 of the spatial levitation image display device 1000, thereby issuing a display request 2030 from the portable terminal 2000 to the spatial levitation image display device 1000. Figure 18A It's a 3D image. Figure 18B This is a top view (XY plan view) taken from above. Figure 18C This is a side view (YZ plan view) taken from the side. Figure 18D This is a front view (XZ plan view) of the device as seen from the front.

[0400] Figure 18A In the case of housing 1190, an opening 1200 is provided on the upper surface, and at the opening 1200, for example, with... Figure 2A Similarly, a transparent component 100 (e.g., a glass plate) and a polarization separation component 101 are also provided. The display range 3R represents a specified position or plane within the space in which the spatially suspended image 3 is formed as a real image. Figure 18A In this context, the spatial coordinate system is represented by (X, Y, Z). The X-axis / X-direction is the left-right direction (first horizontal direction) as seen from user 230 (a viewpoint not shown at the origin of arrow A). The Y-axis / Y-direction is the front-back direction (depth direction, second horizontal direction) as seen from user 230. The Z-axis / Z-direction is the up-down direction (vertical direction) as seen from user 230. Additionally, the coordinate system in the spatially suspended image 3 is represented by (x, y, z). The x-direction is the horizontal direction of the image, the y-direction is the vertical direction, and the z-direction is the depth direction.

[0401] Direction 1801 represents an example of the direction (insertion direction) in which the portable terminal 2000 inserts the display range 3R of the spatial levitation image 3 when the user 230 makes a contact operation. In this example, it is the Y direction. Figure 18A The image shows an example of the contact portion 1802 between the portable terminal 2000 and the display area 3R. In this example, the contact portion 1802 is located near the lower right corner of the display area 3R.

[0402] Figures 18A to 18DIn this embodiment, a display request 2030 is sent from the portable terminal 2000 to the spatial levitation image display device 1000 based on the contact between the portable terminal 2000 and the display area 3R. In other words, the display request 2030 is sent from the portable terminal 2000 to the spatial levitation image display device 1000 before, during, or immediately after contact. For example, the spatial levitation image display device 1000 determines / detects the contact, and upon detection, accepts the display request 2030 sent from the portable terminal 2000. That is, based on the display request 2030, the spatial levitation image display device 1000 allows the display of the target image in the spatial levitation image 3. The spatial levitation image display device 1000 receives the image data of the target image accompanying the display request 2030 from the portable terminal 2000 via transmission. Based on the acquired image data, the spatial levitation image display device 1 controls the display device 1 to display the target image as the spatial levitation image 3. In other words, the spatial levitation image display device 1000 transforms the target image specified by the user 230 into a spatial levitation image and displays it as a spatial levitation image 3.

[0403] The display request 2030 sent from the portable terminal 2000 can use the various communication methods described above, without limitation on the details. In this example, the display request 2030 is sent wirelessly in the LAN 3020, but it is not limited to this. A wired connection can also be used, or the portable terminal 2000 can be connected to the spatial levitation image display device 1000 through infrared communication, etc.

[0404] The determination / detection of contact between the portable terminal 2000 and the display area 3R will be described later. For example, the detection can be performed using an over-the-air sensor 1351, etc. Figure 3 The judgment / detection is performed using the camera unit 1180 (camera). Alternatively, the judgment / detection of the contact can be performed using various sensors installed inside or outside the housing 1190 or various sensors (such as gyroscope sensors) on the portable terminal 2000, or a combination of them.

[0405] As described above, in this system 3000, upon detecting / determining contact between the portable terminal 2000 and the display area 3R, the spatial levitation image display device 1000 recognizes the display request 2030 from the portable terminal 2000, and the spatial levitation image display device 1000 begins displaying the target image of the spatial levitation image. As described above, the feature of Embodiment 3 is that, by simply configuring the portable terminal 2000 in contact with the display area 3R of the spatial levitation image 3, it is possible to display the target image as a spatial levitation image.

[0406] [Contact Operation]

[0407] Figures 19A to 19F The diagram illustrates the orientation change direction when the portable terminal 2000 comes into contact with the display area 3R of the suspended image 3. The orientation change direction 1901 of the portable terminal 2000 is shown in various directions. Figure 19A The diagram uses a 3D representation to show the three-axis rotation angles: pitch angle α, roll angle β, and yaw angle γ. In this embodiment, the orientation of the portable terminal 2000 during touch operation is essentially arbitrary; that is, the states of these angles (α, β, γ) can be arbitrary. Therefore, the touch operation for the user 230 becomes simple and highly convenient.

[0408] Figure 19B This is an XY plane diagram illustrating an example where, during a contact operation, the portable terminal 2000's orientation is rotated within the horizontal plane (XY plane). Here, for illustration, representative positions and orientations of the portable terminal 2000 are set as follows: Figure 19A The front of the housing shown is particularly the central position P1 in the display panel 20012, and the direction R1 along the long side of the front (which is the same as the axis of the roll angle β). In this example, the direction R1 of the portable terminal 2000 upon contact and its corresponding insertion direction 1801 rotate about the Z-axis (yaw angle γ), becoming a direction tilted to the left relative to the Y-axis. The attitude change direction 1901 is the rotation direction about the Z-axis (yaw angle γ). The upper edge of the housing of the portable terminal 2000 contacts the display area 3R (contact portion 1802). The user 230, for example, inserts the portable terminal 2000, held in the right hand, into the vicinity of the center of the display area 3R.

[0409] Figure 19C This is a YZ plane diagram illustrating an example where the portable terminal 2000 rotates within the YZ plane during a contact operation. In this example, the portable terminal 2000's direction R1 and its corresponding insertion direction 1801 are rotated about the X-axis (pitch angle α) during contact, becoming a downward tilt direction relative to the Y-axis. The attitude change direction 1901 is the rotation direction about the X-axis (pitch angle α). This is the case where the insertion direction 1801 is perpendicular to the display area 3R. The upper edge of the portable terminal 2000's housing contacts the display area 3R (contact portion 1802).

[0410] Figure 19D This is an XZ plane diagram illustrating an example where, during a contact operation, the portable terminal 2000's orientation rotates within the XZ plane. In this example, the portable terminal 2000's orientation R1 and its corresponding insertion orientation 1801 rotate about the Y-axis (roll angle β) during contact, resulting in the orientation of the housing's flat plate being vertically upright. The orientation change direction 1901 is the rotational direction about the Y-axis (roll angle β).

[0411] Figure 19E This is a YZ plane cross-sectional view, showing other examples of the orientation of the portable terminal 2000 during a touch operation. This example shows the case where the screen of the front display panel 20012 of the portable terminal 2000 is facing downwards during a touch operation. Figure 19E The application is shown in the image. Figure 4A The structure is used as an example of the structure of the spatial levitation image display device 1000. Image light is emitted obliquely upward from the opening 1200 of the transparent part 100 on the upper surface (XY plane) of the housing 1190 to form a spatial levitation image 3. The plane (xy plane) of the display range 3R of the spatial levitation image 3 is configured to be tilted at about 45 degrees relative to the XY plane. Figure 19E Illustrations of the air operation detection sensor 1351, etc., are omitted (described later).

[0412] When the user 230 touches the screen 1902 downward (direction 1903), the portable terminal 2000 is inserted into the display area 3R in the Y direction (front side) in the insertion direction 1801. Figure 19E The image shows a state where the portable terminal 2000b is in contact with the display area 3R and moved to the deeper side. In this state, the screen 1902 of the portable terminal 2000b is facing downwards (direction 1903). This is the type of contact operation.

[0413] Figure 19F This is a cross-sectional view in the YZ plane, showing other examples of the orientation of the portable terminal 2000 during a touch operation. This example illustrates a situation where, during a touch operation, the screen 1902 of the display panel 20012 on the front of the portable terminal 2000 is oriented along the plane (xy plane) of the display area 3R of the spatially suspended image 3. The user 230 performs an operation that causes the screen 1902 of the portable terminal 2000 to overlap face-to-face with the display area 3R. Figure 19F In the process, the screen 1902 of the portable terminal 2000 is kept in contact with the display area 3R.

[0414] In addition, for the specified contact operation, the time during which the portable terminal 2000 contacts the display range 3R (referred to as the contact time) can be determined. That is, the spatial levitation image display device 1000 measures the contact time and uses it as a judgment condition; if the contact time is longer than a certain period, it is determined that the specified contact operation has been performed.

[0415] Figures 20A to 20CThis is an explanatory diagram illustrating that during the contact operation of inserting the portable terminal 2000 into the display area 3R, the position of the portable terminal 2000 can be virtually arbitrary at various points in time, including before, during, and after insertion. Regarding the insertion of the portable terminal 2000 into the display area 3R, it is sufficient for the portable terminal 2000 to make contact with the display area 3R; in other words, as long as contact can be determined, any position, insertion direction, and angle can be used. Furthermore, contact between the display area 3R and the portable terminal 2000 can essentially occur at any point within the display area 3R.

[0416] In this way, by expanding the range of allowed contact operations / actions (corresponding judgment conditions), it has the advantage of making user operation easier. Conversely, it is also possible to narrow the range of allowed contact operations / actions (judgment conditions). That is, the position, posture, insertion direction, contact time, and contact area of ​​the display range 3R of the portable terminal 2000 can be limited during contact operations (before insertion, during insertion, after insertion, etc.).

[0417] Figure 20A Examples of the position and insertion direction of the portable terminal 2000 before insertion are shown in the YZ plane. Figure 20A The illustration of the portable terminal 2000 is omitted; its position is indicated by dots. Black and white dots (p1, etc.) represent examples of the portable terminal 2000's position before insertion. As shown, the portable terminal 2000 only needs to be in contact with the display area 3R, and various positions and insertion directions are possible.

[0418] Figure 20B Various examples of the position and insertion direction of the portable terminal 2000 before insertion are shown in the XY plane. It can be operated from any position before insertion (p11, etc.) in any insertion direction to bring it into contact with the display area 3R. Furthermore, in the case of position p14, before insertion, the portable terminal 2000 is located to the right of the display area 3R at its depth in the Y direction, and the insertion direction is either from the depth (+Y) towards the near (-Y) direction relative to the display area 3R, or a left-right direction. In this case, as long as the portable terminal 2000 is in contact with the display area 3R, a contact operation is allowed.

[0419] Figure 20C A YZ plane diagram illustrates an example of the state when the portable terminal 2000 is in contact with the display area 3R. Regarding the contact between the portable terminal 2000 and the display area 3R, any contact that can be detected and determined to have occurred is acceptable; it can be contact between any part of the portable terminal 2000. Furthermore, after the portable terminal 2000 has made contact with the display area 3R (after the contact operation is detected), the portable terminal 2000 can move away from the display area 3R.

[0420] State 2000c1 represents an example of the state of the portable terminal 2000 during contact. In this state, the contact area (contact position) c1 is near the center of one side of the display area 3R, specifically a portion near the upper edge of the casing on one side of the portable terminal 2000. State 2000c2 represents another example of the state of the portable terminal 2000 during contact. In this state, the contact area c2 is the upper part of the display area 3R and a portion near the lower edge of the casing, for example. State 2000c3 represents another example of the state of the portable terminal 2000 during contact. In this state, the contact area c3 is the lower part of one side of the display area 3R. After initial contact, the portable terminal 2000 moves to a deeper position and no longer contacts the display area 3R. Furthermore, state 2000c4 represents the portable terminal 2000 returning to its near-side position in the Y direction from the contact state 2000c1.

[0421] [Variation: Judgment Condition]

[0422] In Example 3, the posture and position of the portable terminal 2000 during insertion / contact operations on the display area 3R are arbitrary, but not limited to this. In a modified example, the posture and position of the portable terminal 2000 during insertion / contact operations can be limited to a specific state. That is, when the portable terminal 2000 is contacted, the spatial levitation image display device 1000 can determine / detect a specific posture or other state, and only allow the display request 2030 if detected. As a modified example, when the allowed contact operations are limited to a narrower range, the following judgment conditions can be cited as an example.

[0423] (1) Only the following insertion direction is allowed: For the plane of the display area 3R of the spatial levitation image 3, insertion is allowed in the Y direction from the front, i.e., from the side where the user 230 views the spatial levitation image 3, towards the depth. In other words, contact with the display area 3R from the rear or side is not allowed. In addition, only cases where the insertion direction is within a specified angle range relative to the plane of the display area 3R of the spatial levitation image 3 are allowed.

[0424] (2). When the portable terminal 2000 touches the plane of the display area 3R, the contact point is defined as a part of the position of the portable terminal 2000 and the area within the display area 3R. For example, it is a part of the center, a part of the lower right, a part of the upper left, etc. within the display area 3R.

[0425] (3) The posture of the portable terminal 2000 when it contacts the plane of the display area 3R is limited to a certain range. For example, the angle relative to the plane of the display area 3R is within a specified angle range. For example, only postures such as Figure 19CAs shown, the flat casing of the portable terminal 2000 is inserted approximately vertically into the plane of the display area 3R. Additionally, for example, it may only allow... Figure 19F As shown, the flat casing of the portable terminal 2000 is in contact with the plane of the display area 3R in a roughly parallel manner.

[0426] (4) The posture and holding time of the portable terminal 2000 after it has moved to the deeper side after contacting the plane of the display area 3R. For example, such as Figure 19E As shown in the example, the attitude angle relative to the plane of the display range 3R or the opening 1200 is within a specified angle range, etc.

[0427] [Guide Display]

[0428] Figure 21A This example illustrates how, when a user inserts a portable terminal 2000 into the display area 3R of the levitation image 3, the levitation image display system displays guide information about the insertion / contact operation to the user 230. In Embodiment 3, the levitation image display device 1000 displays this guide information within the levitation image 3; in other words, it guides / instructs the user on the insertion point, direction, and user interface information of the portable terminal 2000. Thus, the user 230 can easily perform the contact operation of inserting the portable terminal 2000 into the display area 3R according to this guide information.

[0429] The following examples can be used as examples of guidance displayed in this guidance information.

[0430] (a) Display the entire display area 3R (the area that receives touch operations) using a specified color, etc. Alternatively, display a frame representing the entire display area 3R using a color, etc.

[0431] (b) Use arrows or other markings to indicate the insertion location and direction.

[0432] (c) Use natural language or images, pictures, animations, etc. to describe the insertion location and direction.

[0433] Figure 21A In the example, the xy-plane diagram showing the guidance information is illustrated when the spatial levitation image 3 is viewed from the bottom front. Figure 21AThe guidance information includes several guidance display examples, and at least one of them needs to be used. Guidance display example 2101 is an example of displaying a rectangle representing the entire display area 3R in color. Guidance display example 2102 is an example of showing the area to be inserted / touched to the user by displaying an arrow image (or other specified symbols or graphics, etc.). In this example, the arrow image is an image of multiple arrows pointing inward from the four sides of the display area 3R. Guidance display example 2103 is an example of informing the user to insert / touch the area of ​​the display area 3R by displaying a text image in natural language (e.g., "Insert here"). Guidance display example 2104 is an example of informing the user to insert / touch the area of ​​the display area 3R by using an image / video, such as an animation, etc.

[0434] As a variation, the guidance information can also be displayed on the screen on one side of the portable terminal 2000, as described later.

[0435] Alternatively, as a variation, a guide image notifying the contact status can be displayed in the spatial floating image 3 during the insertion / contact operation of the portable terminal 2000. Figure 21B In the example, only a local area 2105 in the display range 3R of the spatial levitation image 3 is defined as the designated area (insertion area / contact area) for accepting insertion / contact operations.

[0436] The spatial levitation image display device 1000 uses sensors to detect the contact status of the portable terminal 2000 with the display area 3R, and displays a guide image 2106 indicating the contact status based on the detected contact point. This guide image 2106 indicates that the portable terminal 2000 has made contact with the display area 3R at a spatial position corresponding to the guide image 2106. In this example, the guide image 2106 is a wavy effect image spreading out from the contact point. By viewing the guide image 2106, the user 230 can more easily understand the contact status between the portable terminal 2000 and the display area 3R. Therefore, the user 230 can move the portable terminal 2000 to the local area 2105, i.e., the insertion area, based on the guide image 2106. The guidance information / feedback information provided to the user 230 is not limited to the image display described above; sound output, etc., can also be used simultaneously.

[0437] [Display restriction function]

[0438] In addition, the system 3000 may have the following function (referred to as the display limitation function). When the portable terminal 2000 is inserted into the display area 3R during a touch operation, the system 3000 can prevent the display of the spatial floating image 3 in a localized area near the column where the portable terminal 2000 is inserted within the display area 3R, and allow other displays to be performed in the remaining area of ​​the display area 3R. With the insertion of the portable terminal 2000, when the user 230 views the display area 3R, there may be areas where the displayed image, i.e., the spatial floating image 3, is no longer visible due to light blocking / obstruction by the portable terminal 2000. This function controls the temporary non-display of the spatial floating image 3 in these unvisible areas (localized areas). In this function, the spatial floating image display device 1000, especially... Figure 3 The image control unit 1160 and the like can achieve the above-mentioned non-display by controlling the light source device 13 or the image display element (liquid crystal display panel 11) of the display device 1.

[0439] In addition, when a local area of ​​the display range 3R is not displayed due to the above-mentioned contact operation, the spatial levitation image display device 1000 can reduce the display size of the display image or change the aspect ratio in order to match the remaining display area with the display image, i.e., the spatial levitation image 3.

[0440] Figures 22A to 22C This represents an example of the display limitation function. Figure 22A The XY plane diagram shows, for example, with Figures 18A to 18D Similarly, user 230 inserts portable terminal 2000 into a portion of display area 3R in the Y direction until it makes contact. Area 2201 indicates the position and area of ​​portable terminal 2000 when it is in contact after being inserted into display area 3R. Figure 22B Indicates and Figure 22A The XZ-plane diagram corresponding to the state. Region 2202 is an example of an invisible region caused by occlusion of image light. Additionally, correspondingly, Figure 22C The image light occlusion state is shown in the YZ plane diagram. Position 2203 indicates the position and area of ​​the portable terminal 2000 in the contact state after insertion into the display area 3R.

[0441] Image light component 2204 represents the portion of the image light beam emitted from the opening 1200 that is not blocked by the portable terminal 2000 (position 2203) and thus enters the viewpoint (eye) of the user 230, forming a spatially suspended image 3 within the display area 3R. Image light component 2205 represents the portion of the image light beam emitted from the opening 1200 that is blocked by the portable terminal 2000 (position 2203) and thus does not enter the viewpoint (eye) of the user 230, forming an invisible area 2202 within the display area 3R.

[0442] Figures 23A-23B Examples of displaying the guide image in the spatial levitation image 3 and examples of display restriction function are shown.

[0443] first, Figure 23A An example is shown in the front view (xy plane) of the display area 3R of the spatial levitation image 3, in which, upon detection of a touch operation, a guide image 2302 indicating that a display request 2030 is being generated is displayed. In this example, within the display area 3R, there is a border label 2301 representing the entire area, within which the guide image 2302 indicating that a display request 2030 is being generated is displayed. The guide image 2302 is, for example, a text image indicating that a display request 2030 is being generated, such as "Requesting..." or an arrow icon. Furthermore, in this state, an invisible area 2202 is created due to the obstruction of the portable terminal 2000.

[0444] Figure 23B The space-suspended image display device 1000 is shown based on the detection of generated images such as Figure 23A The example shown is an invisible area 2202, which causes a local area in the display range 3R to be undisplayed and controls the display of the remaining area. The generation of the invisible area 2202 can be detected by the air operation detection sensor 1351 or the camera unit 1180.

[0445] For example, the spatial levitation image display device 1000 Figure 3 The image control unit 1160 determines / detects contact operations and identifies / detects the invisible area 2202, determining a non-display area within the display range 3R that includes the invisible area 2202. In this example, based on the detection of the air operation detection sensor 1351 (described later), the contact portion 2303 in the X and x directions of the display range 3R is identified, in other words, its width and column. Therefore, the spatial levitation image display device 1000 determines the display column in the display range 3R corresponding to the contact portion 2303 as a non-display area 2304 represented by a diagonal shading pattern. The non-display area 2304 includes the invisible area 2202. The area in the display range 3R other than the non-display area 2304 is the display area 2305. In other words, the spatial levitation image display device 1000 reduces the size of the display range 3R in the X and x directions by an amount equivalent to the width of the non-display area 2304, thereby constructing the display area 2305.

[0446] The spatial levitation image display device 1000 controls the display of the display device 1 by not displaying the spatial levitation image 3 in the non-display area 2304, but only displaying the spatial levitation image 3 in the display area 2305. In this example, in the display area 2305 of the display range 3R, from... Figure 23A The original border image 2301 and guide image 2302 shown are reduced in size and adjusted to fit within the width of the display area 2305 while maintaining the aspect ratio, thus becoming border image 2306 and guide image 2307.

[0447] Through the aforementioned functions, user 230 can easily view the guide image 2307, etc., in the spatial floating image 3. The above example illustrates the case where an area larger than the invisible area 2202 is designated as the non-display area 2304, but it is not limited to this. The non-display area can also be the same area as the invisible area, or it can be a different area that at least partially overlaps with the invisible area. In addition, various changes and adjustments to the guide image can include changes in aspect ratio, color or brightness, and shifting of the display position.

[0448] Figure 24A This example illustrates a display of a guide image for the spatial levitation image 3, shown as feedback to the user 230 when a specified contact operation is detected, or in other words, when the contact operation is deemed successful and the display request 2030 is accepted. In this example, a success guide image 2401 is displayed across the entire display area 3R. The guide image 2401 consists of text and icons indicating success. Upon such success, the process proceeds to the stage of displaying the target image as the spatial levitation image 3.

[0449] Figure 24B This is an example of a guide image displayed as feedback to the user 230 when the specified contact operation was not detected, or in other words, the contact operation was deemed to have failed and the display request 2030 was not accepted. In this example, a guide image 2402 indicating failure is displayed throughout the entire display area 3R. The guide image 2402 consists of text, icons, etc., indicating failure and retry.

[0450] If the user 230's insertion / contact operation with the portable terminal 2000 to the display area 3R fails to be performed properly, or if the operation is performed properly but the system 3000 fails to detect it properly due to sensor malfunction or other reasons, the spatial levitation image display device 1000, as described above, determines it as a failure or error and outputs a guidance image 2402 to the user 230 as feedback. In this case, the spatial levitation image display device 1000 can remind the user 230 to retry the contact operation. The user 230 then performs the contact operation again.

[0451] and then, Figure 24CIn an example where a guide image indicating that the portable terminal 2000 has appropriately touched the display area 3R and that the portable terminal 2000 has successfully issued a display request 2030 to the spatial levitation image display device 1000 is displayed as a spatial levitation image 3, the display area is adjusted according to the invisible area 2202 for display. The non-display area 2404 is the area within the display area 3R that includes the invisible area 2202, and the display area 2405 is the area outside the non-display area 2404.

[0452] Figure 24C The lower left (A) is an example of a successful case, displayed in display area 2405 based on... Figure 24A The guide image 2401b shown is an adjusted version of the guide image 2401, including resizing. Similarly, Figure 24C The lower right side (B) shows an example of a failure scenario, displayed in display area 2405 based on, for example... Figure 24B The guide image 2402b shown is the guide image 2402 after adjustments such as size.

[0453] The feedback to user 230 described above is not limited to visual means such as displaying guide images in the spatial levitation image 3 or illuminating indicator lights; it can also use auditory means such as sound output or tactile means such as devices that generate vibrations / air / ultrasound. Such devices, such as air jets or ultrasonic generators, can be installed inside or outside the housing 1190. Furthermore, various means can be used in combination simultaneously.

[0454] Figure 24D In the YZ plane, such a situation is illustrated, where, in the case of success or failure of the contact operation and display request 2030, the spatial levitation image display device 1000 responds to the user 230 via speakers, etc. Figure 3 The sound output unit 1140 emits a sound 2408 indicating success or failure. (A) is the sound 2408 in the case of success (sound notification example), such as "connected". (B) is the sound 2408 in the case of failure (sound notification example), such as "unable to connect" or "please try again". The speaker can be installed inside or outside the housing 1190. The speaker can be a super-directional speaker, etc. The above example is a natural language sound output example, but it is not limited to this; it can also output prescribed short tones such as beeps, alarm tones, or music.

[0455] [Over-the-air operation detection sensor]

[0456] Figures 25A-25CThis illustrates an example of the installation of the aerial operation detection sensor 1351 and the camera unit 1180 (camera) as sensors in the space-floating image display device 1000 of Embodiment 3. The space-floating image display device 1000 of Embodiment 3 is particularly... Figure 3 The aerial operation detection unit 1350, control unit 1110, or image control unit 1160, etc., use these sensors to determine / detect the insertion / contact operation performed by the portable terminal 2000 on the display range 3R of the spatial levitation image 3.

[0457] Figure 25A The YZ plan view shows an example of the configuration of the air operation detection sensor 1351 and the camera unit 1180 (camera) in the housing 1190. Figure 25A Examples of optical system structures and Figure 4A Same as Figure 18E. Figure 25A In this example, the airborne operation detection sensor 1351 is positioned near the transparent component 100 on the upper surface of the housing 1190, close to the user 230 in the Y direction. In this airborne operation detection sensor 1351, the optical axis, indicated by a single-dotted arrow, points diagonally upwards in the Y direction, and this optical axis is set to coincide with the display range 3R of the spatial levitation image 3. Furthermore, Figure 25A In this example, the camera unit 1180 (camera) is configured within the housing 1190 such that the optical axis, indicated by the single-dotted arrow, points upwards, i.e., in the Z direction. The shooting range of the camera unit 1180 (camera), as shown by the dashed line, is set to include the display area 3R and the opening 1200.

[0458] Figure 25B This section illustrates the structure of the airborne operation detection sensor 1351. Figure 25B The image shows an xy-plane diagram corresponding to the display range 3R of the space-suspended image 3. The aerial operation detection sensor 1351 is located in the x-direction (…). Figure 25A Multiple optical elements 1351c are arranged in the X direction (of the sensor). Each optical element 1351c is a pair of light-emitting elements 1351a and light-receiving elements 1351b. The light-emitting element 1351a is, for example, composed of an infrared element. The emission surface of the light-emitting element 1351a of the airborne operation detection sensor 1351 is... Figure 25A The upper surface of the transparent component 100 of the housing 1190 is consistent. The light-emitting element 1351a emits light a1 (e.g., infrared light) in the y direction. This light a1 passes through the display area 3R when it is not blocked by an object. When the light a1 is blocked by an object, it is reflected by the object and returns as reflected light a2. The reflected light a2 is received by the light-receiving element 1351b.

[0459] For example, if the user 230's finger touches a contact point 2501 within the xy plane of the display area 3R, light a1 is reflected at the contact point 2501 and returns as reflected light a2. A light-receiving element 1351b at a certain x-direction position detects this reflected light a2. Therefore, the air-operation detection unit 1350 can determine that the contact point 2501 is located at that x-direction position. Furthermore, the air-operation detection unit 1350 can calculate the distance using a TOF method based on the time it takes for the emitted light a1 to return as reflected light a2. For example, it can calculate the distance 2502 to the contact point 2501. Thus, the position coordinates of the contact point 2501 on the xy plane of the display area 3R can be determined.

[0460] in addition, Figure 25B In the diagram, the portable terminal 2000b is shown positioned within the display range 3R as the contact point 2503 during an insertion / contact operation. In this case, the air operation detection sensor 1351 and the air operation detection unit 1350 can also detect the position coordinates of the contact point 2503. Furthermore, Figure 25A In this example, there are no objects above the suspended image 3, but a portion of the housing 1190 can be placed above the suspended image 3. In this case, the light a1 from the airborne operation detection sensor 1351 is reflected by a portion of the housing 1190, becoming reflected light a2 and returning.

[0461] The space-based levitation image display device 1000 uses the aforementioned airborne operation detection sensor 1351 and airborne operation detection unit 1350, as in the portable terminal 2000b. Figure 25A When the display area 3R is touched, a contact operation can be detected. Not limited to the above configuration example, the air operation detection sensor 1351 can be configured on the upper side relative to the xy plane of the display area 3R, or it can be configured at a position offset in the z-direction and y-direction, or multiple air operation detection sensors can be configured at multiple positions in the z-direction and y-direction.

[0462] in addition, Figure 25A In the example where the portable terminal 2000b is positioned within the display area 3R, the portable terminal 2000b will appear within the image captured by the camera unit 1180 (camera). Therefore, the image control unit 1160 and the like, which cooperate with the camera unit 1180, can detect the portable terminal 2000b within the captured image and determine whether the portable terminal 2000b is in contact with the display area 3R. Thus, the spatial levitation image display device 1000 uses the aforementioned camera unit 1180 to display the portable terminal 2000b within the display area 3R. Figure 25AEven when the display area 3R is touched, the touch operation can still be detected. The spatial levitation image display device 1000 can use at least one of the above-mentioned air operation detection sensor 1351 and camera unit 1180 to determine / detect the touch operation. Using both of these can further improve the determination / detection accuracy.

[0463] Not limited to Figure 25A For example, as another configuration of the camera unit 1180 (camera), as shown by the dashed line 1180b, it can be configured such that its optical axis points in the direction of the display range 3R from a deeper position in the Y direction outside the housing 1190. The camera unit 1180b can capture images of the user 230's face, and can also detect when the user 230 is approaching. Furthermore, it is not limited to... Figure 25B The aerial operation detection sensor 1351 shown senses the plane parallel to the plane of the spatial levitation image 3. Alternatively, a range sensor, stereo camera, or the like that senses the plane from a direction perpendicular to the plane of the spatial levitation image 3 can also be used.

[0464] Figure 25C Other configuration examples of the air operation detection sensor 1351 and the camera unit 1180 (camera) in the housing 1190 are shown. Figure 25C Examples illustrate such a situation where the optical system is... Figure 2B Based on this, the housing 1190 is arranged longitudinally to vertically form a spatial levitation image 3 in the vertical direction (Z direction). The components of the aforementioned optical system are mounted at the rear of the housing 1190 in the Y direction, and the spatial levitation image 3 is formed between the upper and lower housing portions at the front of the housing 1190. In this example, an air operation detection sensor 1351 and a camera unit 1180 (camera) are provided on the upper housing portion. The optical axis of the air operation detection sensor 1351 points downwards, coinciding with the display area 3R. The shooting range of the camera unit 1180 (camera) is set to include the display area 3R. With this structure, it is also possible to detect contact operations using at least one of the air operation detection sensor 1351 and the camera unit 1180 (camera) when the portable terminal 2000 is inserted into the display area 3R.

[0465] [Regarding the distinction between fingers and portable devices]

[0466] The spatial levitation image display device 1000 can distinguish whether the object in the spatial levitation image 3 (display area 3R) is the user 230's finger or the portable terminal 2000, or it can choose not to distinguish. If it does distinguish, the following technical means can be cited as examples.

[0467] (1) The spatial levitation image display device 1000 can identify the portable terminal 2000 based on communication information such as connection requests during arbitrary communication between the portable terminal 2000 and the spatial levitation image display device 1000. When the spatial levitation image display device 1000 detects that an object has come into contact with the display range 3R through the air operation detection sensor 1351, it can presume that the object is the portable terminal 2000 if the aforementioned communication information is present.

[0468] (2). The spatial levitation image display device 1000 can identify and determine that the object in contact with the spatial levitation image 3 is the portable terminal 2000 based on the images captured by the camera unit 1180 (camera) and the like through image recognition processing.

[0469] (3) The spatial levitation image display device 1000 can use the air operation detection sensor 1351 and the air operation detection unit 1350 to determine whether the object in contact with the spatial levitation image 3 is the user 230's finger or the portable terminal 2000. For example, as a typical air operation or touch operation, the operation received is performed by one finger (e.g., detecting...). Figure 25B The contact area 2501). When the portable terminal 2000 is inserted into the display area 3R, the area of ​​the contact area of ​​the portable terminal 2000 is larger than a finger, and the shape of the contact area is linear, which can be used to distinguish it. The spatial levitation image display device 1000 uses an air operation detection sensor 1351 and the like to detect and determine the contact area (e.g., Figure 25B (Based on the area, shape, etc. of the contact part 2503), it is determined whether the object being contacted is a finger or a portable terminal 2000.

[0470] (4) The information transmitted between the spatial levitation image display device 1000 and the portable terminal 2000 includes information representing the portable terminal 2000 (e.g., a QR code described later). By recognizing and obtaining this information, the spatial levitation image display device 1000 can distinguish whether the object in contact with the spatial levitation image 3 is the user 230's finger or the portable terminal 2000.

[0471] [Applications and screen of portable terminals]

[0472] Figure 26AThis illustrates a display example where the application 2010, etc., from the portable terminal 2000 is displayed on the screen of the display unit 20012 in Embodiment 3. Based on input operations from the user 230, the portable terminal 2000 displays an application screen 2601 on the screen of the display unit 20012 through processing by the OS and the application 2010, etc. In this example, the application 2010 is a dedicated application that works in conjunction with the spatial levitation image display device 1000 to enable the user 230 to conveniently use various functions of the spatial levitation image display device 1000.

[0473] Figure 26A In application screen 2601, there is a GUI that prompts user 230 to specify the image 2020 (target image) to be displayed in the spatial levitation image 3 of the spatial levitation image display device 1000. For example, a message "Please specify the image to be displayed on the aerial display" is displayed. User 230 selects the image 2020 to be displayed on the aerial display from the images stored in the portable terminal 2000. Figure 15 The selected image is previewed and displayed as the target image 2602. After confirming the target image 2602, the user 230 presses the display request button 2603. As a result, the portable terminal 2000 sends a display request 2030 with the target image 2602 as the target image to the spatial levitation image display device 1000 via near-field wireless communication or other means.

[0474] Figure 26B It is about the above. Figure 21A A variation of the guide image display is shown on a screen on one side of the portable terminal 2000. Figure 26B In the application screen 2601, a guide image 2604 is displayed, reminding the user 230 to perform a contact operation. The guide image 2604 displays, for example, a message that says "Please make your smartphone contact the floating image in the air," and also shows an image indicating the action of inserting the portable terminal 2000 into the display area 3R to make contact.

[0475] Figure 26C It is about the above. Figure 23A The guide image is displayed and Figure 24A A variation of the guide image display is shown on a screen on one side of the portable terminal 2000. Figure 26C In the application screen 2601, a guide image 2605 is displayed to notify the user 230 that a display request 2030 is being sent, or a guide image 2606 is displayed to notify the user 230 that the display request 2030 has been successfully accepted, depending on the situation.

[0476] In Embodiment 3, one method is that, during a touch operation, as described above, the user 230 specifies the target image 2020 on the screen of application 2010 on the portable terminal 2000 and presses the display request button 2603, and the portable terminal 2000 sends a display request 2030. Afterwards, the user 230 inserts the portable terminal 2000 into the display area 3R.

[0477] Not limited to this, in other embodiments, the operation of user 230 can be reduced to a minimum. For example, user 230 causes the screen of portable terminal 2000 to display a desired target image, and in this state, portable terminal 2000 is inserted into display area 3R. Spatial levitation image display device 1000 determines / detects the contact operation, and generates display request 2030 if successful. Spatial levitation image display device 1000 communicates with portable terminal 2000 to obtain the target image associated with display request 2030, and displays it in spatial levitation image 3. This target image is the image currently displayed on the screen of portable terminal 2000.

[0478] [Display of target image in spatial levitation image]

[0479] Image control unit 1160 of spatial levitation image display device 1000 Figure 3 Based on the image data of the target image, image data for display on the screen of the display device 1 (liquid crystal display panel 11) is generated. For example, if the target image is two-dimensional image data, the image data is two-dimensional image data generated by processing the two-dimensional image data. In addition, if the target image is three-dimensional image data (three-dimensional model, etc.), the image data is two-dimensional image data generated by rendering in a virtual three-dimensional space, etc., based on the three-dimensional image data.

[0480] Through the above functions, for example, Figure 16 As shown, when user 230 displays and specifies a target image 1601 (an example of a character image) using portable terminal 2000, the image 1602 can be displayed as a spatial levitation image 3 in response to a touch operation and display request 2030. Thus, user 230 can view and appreciate the requested target image 1601, etc., through the image 1602 of the spatial levitation image 3.

[0481] The method for de-displaying the target image in the spatial levitation image 3 is not limited. For example, it can be automatically de-displayed after a certain period of time after the target image is initially displayed. Alternatively, a display de-display button can be provided on the screen of the spatial levitation image 3 or the portable terminal 2000. A display de-display button can also be provided on the housing 1190.

[0482] As described above, the spatial levitation image display system according to Embodiment 3, in cooperation with the portable terminal 2000, can provide a service to the user 230 that can display the requested image as a spatial levitation image 3 for viewing.

[0483] Furthermore, the data / information transmitted between the portable terminal 2000 and the spatial levitation image display device 1000 may not be limited to the target image being displayed; it may also transmit certain management / control data / information (such as user information) along with the target image being displayed.

[0484] As a variation, the same method can be achieved, that is, the background image, character image, etc. are displayed as a spatial floating image 3 in advance, and in this state, in response to the user 230's contact operation and display request 2030 of the portable terminal 2000, the target image is superimposed on the background image, etc.

[0485] In Example 3, the contact between the portable terminal 2000 and the spatial levitation image 3 is considered successful, and the display request 2030 is accepted and the target image is displayed, but this is not limited to this. A variation could also be implemented such that, after the user 230 inserts the portable terminal 2000 into the spatial levitation image 3 and makes contact, the portable terminal 2000 passes through the spatial levitation image 3 and enters a deeper space, becoming a state that is not in contact with the display area 3R (e.g., ...). Figure 19E If the result is below 0, it is considered a success.

[0486] Furthermore, a modified example can also be described as follows. In this modified example, the image displayed as the spatial levitation image 3 is acquired on the portable terminal 2000 side. The spatial levitation image display device 1000 pre-displays character images, etc. (e.g., with...) in the spatial levitation image 3. Figure 16 (same as (3)). When user 230 wants to obtain the image that is being displayed as the spatial levitation image 3 on the portable terminal 2000, the portable terminal 2000 performs a contact operation with the spatial levitation image 3 in the same manner as described above. When the contact operation is successful, the spatial levitation image display device 1000 sends the image that is being displayed as the spatial levitation image 3 (and its corresponding image data) to the portable terminal 2000. Thus, user 230 can obtain, display, view and appreciate the image on the portable terminal 2000 (e.g., with the image displayed as the spatial levitation image 3). Figure 16 (1) is the same).

[0487] In Example 3, a function is employed whereby, after the portable terminal 2000 makes contact with the spatial levitation image 3 to display the target image, the target image continues to be displayed even when the portable terminal 2000 no longer makes contact with the spatial levitation image 3. However, this is not a limitation. A modified example could employ a function that displays the target image only while the portable terminal 2000 is continuously in contact with the spatial levitation image 3. The spatial levitation image display device 1000 deactivates the display of the target image in the spatial levitation image 3 when it detects that the portable terminal 2000 has left the display range 3R.

[0488] <Example 4>

[0489] Next, as another embodiment (referred to as Embodiment 4), using Figure 27A The accompanying drawings illustrate examples using QR codes (two-dimensional codes). Embodiment 4 is a variation of Embodiment 3. The portable terminal 2000 has the function of appropriately displaying a QR code on the screen of the display unit 20012. This QR code may, for example, be a barcode representing the aforementioned display request 2030.

[0490] In Example 4, when a display request 2030 is made from the portable terminal 2000 to the spatial levitation image display device 1000, the following operations / actions can be performed for the purpose of enabling the portable terminal 2000 to recognize the spatial levitation image display device 1000, or enabling the spatial levitation image display device 1000 to recognize the portable terminal 2000. The user 230 operates from the application 2010, etc., on the portable terminal 2000 to request cooperation, connection, and make display requests 2030 (e.g., with...) Figure 26A (The same applies). Based on this, user 230 performs a contact operation to insert portable terminal 2000 into display area 3R. Alternatively, for example, user 230 displays a QR code on the screen of portable terminal 2000 before insertion, and then performs a contact operation to insert portable terminal into display area 3R in a posture that allows the camera unit 1180 (camera) on one side of spatial floating image display device 1000 to read the QR code of portable terminal 2000.

[0491] Figure 27A This is an example of a portable terminal 2000 displaying a QR code image 2701 on the screen 1902 of the display unit 20012 based on the input operation of the user 230, and displaying image information such as QR code 2702 on the QR code image 2701.

[0492] Figure 27B An example is shown in the YZ plane diagram, in which user 230 causes portable terminal 2000a, located before the insertion of display range 3R, to display a QR code on screen 1902. Figure 27AThe portable terminal 2000a is inserted into the display area 3R from this position, becoming the portable terminal 2000b at the contact position. In this state, the screen 1902 of the portable terminal 2000b faces downwards and is within the shooting range of the camera unit 1180 (camera). Therefore, the spatial levitation image display device 1000 can detect the portable terminal 2000b based on the captured image and can detect the QR code 2702 displayed on the screen 1902. The spatial levitation image display device 1000 recognizes the QR code and extracts the data / information recorded in the QR code. For example, the spatial levitation image display device 1000 obtains a display request 2030 from the QR code.

[0493] The levitation image display device 1000 uses an airborne operation detection sensor 1351 or a camera unit 1180 to detect contact between the portable terminal 2000b and the display area 3R, and obtains a display request 2030 from the QR code of the portable terminal 2000b. Based on these actions, the levitation image display device 1000 successfully completes the process and displays the target image associated with the display request 2030 in the levitation image 3.

[0494] in addition, Figure 27B The portable terminal 2000b makes contact with the display area 3R, but is not limited thereto. As mentioned above, the portable terminal 2000 can move into the deeper part of the display area 3R after contacting it, thus entering a non-contact state. The posture of the portable terminal 2000 only needs to be one that can be detected by the camera unit 1180 (camera) of the QR code on the screen 1902. For example, in Figure 25A In the case of the camera unit 1180b, the orientation of the portable terminal 2000 only needs to be such that the screen 1902 faces the deeper side in the Y direction (for example, it can be with...). Figure 19E same).

[0495] In Example 4, as described above, the condition of recognizing a QR code is added to the judgment condition of the contact operation specified in Example 3. In other words, in Example 4, when the AND condition of detecting the specified contact operation and recognizing the QR code is met, the spatial levitation image display device 1000 accepts and allows the display request 2030 and displays the target image in the spatial levitation image 3.

[0496] Example 4 ( Figure 27B In this configuration, user 230 can position the portable terminal 2000 with screen 1902 facing downwards, and insert the portable terminal into the display area 3R (within) in a horizontal direction parallel to the upper surface of the housing 1190. Figure 19E(Same). Additionally, in a modified example, user 230 can position the portable terminal 2000 with screen 1902 facing downwards, and after inserting the display area 3R, place the screen 1902 in contact with and position it against the upper surface of the transparent component 100 of the housing 1190. QR codes can also be recognized in these cases.

[0497] [QR code]

[0498] The data / information recorded in the aforementioned QR codes / barcodes (including one-dimensional and two-dimensional barcodes) is not limited to the examples in Display Request 2030 above; for example, the following can be cited. This data / information can also be used in combination.

[0499] (1). Connection information: The portable terminal 2000 and the spatial levitation image display device 1000 are connected in... Figure 15 Information used for communication connections in a communication network. This includes, for example, IP addresses, IDs, and passwords. This communication network could be the Internet 3010, LAN 3020, or direct communication between the portable terminal 2000 and the spatial levitation image display device 1000.

[0500] (2). Terminal ID / User ID: Information such as the ID of portable terminal 2000. Or information such as the ID of user 230.

[0501] (3). Image ID: The ID or URL of the target image that user 230 wants to display.

[0502] When the connection information is formed into a QR code, the portable terminal 2000 presents the QR code to the spatial levitation image display device 1000 to notify the connection information, and uses the connection information to establish a wireless connection between the communication unit 20020 of the portable terminal 2000 and the communication unit 1132 of the spatial levitation image display device 1000.

[0503] When the aforementioned terminal ID / user ID is formed into a QR code, the spatial levitation image display device 1000 can identify a specific portable terminal 2000 / user 230. The spatial levitation image display device 1000 can allow only the specific portable terminal 2000 / user 230 associated with the specific terminal ID / user ID to become the service provider for this function.

[0504] When the above image ID is formed into a QR code, the target image is not limited to the image stored in the portable terminal 2000.

[0505] As described above, according to the spatial levitation image display system of Embodiment 4, by using the above-mentioned QR code, it is possible to specify the prescribed contact operation between the portable terminal 2000 and the spatial levitation image display device 1000, and to perform various controls using the above-mentioned QR code.

[0506] In addition, in embodiments such as Example 3 that do not use QR codes, the data / information described above can be transmitted and used together with the display request 2030 between the portable terminal 2000 and the spatial levitation image display device 1000.

[0507] As a variation of Embodiment 4, the spatial levitation image display device 1000 may send the aforementioned QR code information to a nearby portable terminal 2000, and the portable terminal 2000 that receives the information may display the aforementioned QR code.

[0508] In embodiments 3 and 4, the data of the target image to be displayed in the spatial levitation image 3 is pre-stored in the portable terminal 2000, but this is not a limitation. In a variation, the target image data may be stored on the spatial levitation image display device 1000 side, and the user 230 or the portable terminal 2000 side selects and specifies the target image and sends a display request 2030. Alternatively, in other variations, the target image data may be stored on a server or similar device on a communication network, and the user 230 or the portable terminal 2000 side selects and specifies the target image and sends a display request 2030.

[0509] Alternatively, as a variation of Embodiment 4, the spatial levitation image display device 1000 may display the QR code information in the spatial levitation image 3, and the user 230 may use the camera of the portable terminal 2000 to identify the QR code information displayed in the spatial levitation image 3.

[0510] Figures 28A-28C This indicates the variant example. Figure 28A This is an example of a space levitation image display device 1000 displaying a QR code 2801 as a space levitation image 3. Figure 28B This is a YZ floor plan, which is captured and read by the portable terminal 2000 using a camera (e.g., a rear camera) in the camera unit 20025. Figure 28A The example shown is a QR code 2801 used as a spatial levitation image 3. The portable terminal 2000 obtains the specified information based on the recognized QR code 2801. After recognizing the QR code 2801, the user 230 causes the portable terminal 2000 to touch the display area 3R.

[0511] Figure 28C Another variation is the display of QR code 2802 on the screen of the second display device 1680 of the spatial levitation image display device 1000. The spatial levitation image display device 1000, for example, is... Figure 4M Similarly, using a second display device 1680 that is configured to overlap with the spatial levitation image 3 on the depth side, the QR code 2802 is displayed on the screen of the second display device 1680. Additionally, the spatial levitation image display device 1000 can also be used with... Figure 4L Similarly, a QR code is displayed on the screen of a transmissive self-emissive image display device 1650.

[0512] User 230 uses the camera of the camera unit 20025 of the portable terminal 2000 to capture the QR code 2802 displayed on the screen of the second display device 1680. The portable terminal 2000 recognizes the QR code and obtains information. After recognizing the QR code 2802, user 230 brings the portable terminal 2000 into contact with the display area 3R.

[0513] <Example 5>

[0514] As Embodiment 5 of the present invention, a structural example of a spatial levitation image display device will be described. The spatial levitation image display device of Embodiment 5 can also utilize the structures of Embodiments 1 to 4 as its basic structure. Furthermore, like Embodiments 3 and 4, Embodiment 5 can be a system (spatial levitation image display system) having a spatial levitation image display device and external devices connected thereto. External devices are, for example, portable terminals (mobile information processing terminal devices) such as smartphones, tablets, or wearable terminals (smartwatches, etc.) held by the user.

[0515] [Technical problems and solutions related to Example 5]

[0516] The technical issues are as follows: The sensors included in the space-based levitation image display device 1000 are an airborne operation detection sensor 1351 and an attitude sensor 1113. Figure 3 Various sensors, such as touch sensors and attitude sensors, are used, but the portable terminal 2000 may also have the same types of sensors. As described in Examples 3 and 4, when the portable terminal 2000 and the spatial levitation image display device 1000 are communicatively connected, various possibilities can be considered regarding which side's operation and detection (sensors) should be effective and used. When it is unclear which side's operation and detection should be effective, depending on the purpose, function, and situation, it may lead to less than ideal actions and effects. Therefore, it is necessary to clarify which side's operation and detection should be effective based on the purpose, function, and situation.

[0517] Therefore, the spatial levitation image display system and display method of Embodiment 5 have the following functions. The system of Embodiment 5 uses both a sensor on the portable terminal 2000 side for detecting user operations and a sensor on the spatial levitation image display device 1000 side for detecting user operations. This system can use both the sensor on the portable terminal 2000 side for detecting touch and rotation operations performed by the user on the image displayed on the screen, and the sensor on the spatial levitation image display device 1000 side for detecting touch and rotation operations performed by the user on the image displayed in the spatial levitation image 3. When images / videos are displayed on both the screen on the portable terminal 2000 side and the spatial levitation image 3 side (dual display described later), this system can select, set, and control which side's sensor is enabled / disabled.

[0518] [Example 1 of a technical problem and its solution]

[0519] Figure 37 Example 1 illustrating a technical problem and its solution. First, an image 3701 (e.g., image B) is displayed on the screen 2005 of the portable terminal 2000. Based on this image B, the same image 3702 (e.g., image B) is also displayed on the screen of the spatial levitation image display device 1000 (hereinafter also referred to as the screen for convenience), i.e., the spatial levitation image 3. Figure 29 (Dual display function 2910). One technical problem is that, in such a state, if the user 230 touches image B on the screen 2005 of the portable terminal 2000, for example, how the two images 3701 and 3702 will change, which has been unclear in the past.

[0520] In this embodiment, one solution is as follows. In response to detecting, for example, a touch operation by a user on image 3701 (image B) of the screen 2005 of the portable terminal 2000, the system performs display updates and other processing as a pre-defined process associated with image 3701, and performs synchronization processing to ensure that the display updates are also reflected in image 3702 (image B) on one side of the spatially suspended image 3. Figure 29 (Synchronization control function 2920). Image B on the portable terminal 2000 side changes from image B to image Bb due to display update, and image B on the spatial floating image 3 side changes from image B to image Bb due to display update.

[0521] Conversely, such as Figure 37 As shown in the lower part, when a touch operation is performed on image 3702 (image B) on one side of the spatial floating image 3, this system does not perform display updates or other processing on image 3702, nor does it reflect it on image 3701 (image B) on the side of the portable terminal 2000. In other words, no synchronization is performed on the side of the portable terminal 2000.

[0522] The synchronization control described in this example is based on a setting that enables touch operation detection on the portable terminal 2000 side and disables touch operation detection on the spatial levitation image 3 side. Figure 29 The system implements operation detection and control functions (2930). When two devices are connected in communication, the system sets the post-connection state. Each device operates, detects, and displays according to these post-connection settings. This achieves the following: Figure 37 The effects shown are as follows.

[0523] [Example 2 of a technical problem and its solution]

[0524] Figure 38 Example 2 illustrates a technical problem and its solution. Figure 38 In the example, image 3802 (image A) is displayed on the screen of the spatial levitation image display device 1000, i.e., spatial levitation image 3, and based on image A, the same image 3801 (image A) is also displayed on the screen of the portable terminal 2000. Figure 29 (The dual display function 2910). One technical problem is that, in such a state, if the user 230 touches, for example, the image A of the spatial levitation image 3, how the two images 3801 and 3802 will change, which was previously unclear.

[0525] In this embodiment, one solution is as follows. The system performs display updates and other processing associated with image 3802 based on detected user actions, such as touch operations, on image 3802 (image A) of the spatial levitation image 3. Furthermore, to ensure that these display updates are also reflected in image 3801 (image A) on the portable terminal 2000, synchronization processing is performed. Figure 29 (Synchronization control function 2920). Image A on one side of the spatial levitation image 3 becomes image Ab due to display update, and image A on the side of the portable terminal 2000 becomes image Ab due to display update.

[0526] Conversely, such as Figure 38 As shown in the lower part, when a touch operation is performed on image 3801 (image A) on one side of the portable terminal 2000, this system does not perform any display updates or other processing on image 3801, nor does it reflect it on image 3802 (image A) on one side of the spatial floating image 3.

[0527] The synchronization control described in this example is based on the setting that touch operation detection on the portable terminal 2000 side is invalid, while touch operation detection on the spatial levitation image 3 side is valid. Figure 29The system implements operation detection and control functions (2930). When two devices are connected in communication, the system sets the post-connection state. Each device operates, detects, and displays according to these post-connection settings. This achieves the following: Figure 38 The effects shown are as follows.

[0528] [Example 3 of a technical problem and its solution]

[0529] Figure 39 Example 3 illustrates a technical problem and its solution. Figure 39 In the example, first with Figure 37 Similarly, image 3901 (e.g., image B) is displayed on the screen of portable terminal 2000, and based on image B, the same image 3902 (e.g., image B) is also displayed on the screen of spatial levitation image display device 1000, i.e., spatial levitation image 3. Figure 29 (Dual display function 2910). One technical problem is that, in such a state, if the user 230 rotates, for example, the image B on the screen 2005 of the portable terminal 2000, how the two images 3901 and 3902 will change is something that has been unclear in the past.

[0530] In this embodiment, one solution is as follows. The system detects a user's rotation operation on image 3901 (image B) of the screen 2005 of the portable terminal 2000, for example, by rotating the casing to rotate image 3901. Image B seen by user 230 on screen 2005 becomes, for example, image Bc after being rotated 90 degrees. In response to this detection, the system synchronizes the rotation to also reflect it on image 3902 (image B) on one side of the spatially suspended image 3. Figure 29 (Synchronization control function 2920). The image B of the spatial levitation image 3 seen by user 230 becomes the image Bc after being rotated 90 degrees.

[0531] Or, such as Figure 39 As shown below, this system detects a user's rotation operation on image 3901 (image B) of the screen 2005 of the portable terminal 2000, for example, rotating the image via touch. Image B seen by user 230 on screen 2005 becomes, for example, image Bc rotated 90 degrees. In response to this detection, the system synchronizes the rotation to also reflect it on image 3902 (image B) on one side of the spatially suspended image 3. Figure 29 Synchronization control function 2920).

[0532] The synchronization control described in this example is based on the setting that rotation operation detection and image rotation are valid on the portable terminal 2000 side, and rotation operation detection and image rotation are invalid on the spatial floating image 3 side. Figure 29 The system implements operation detection and control functions (2930). When two devices are connected in communication, the system sets the post-connection state. Each device operates, detects, and displays according to these post-connection settings. This achieves the following: Figure 39 The effects shown are as follows.

[0533] [Example 4 of a technical problem and its solution]

[0534] Figure 40 Example 4 illustrates a technical problem and its solution. Figure 40 In the example, first with Figure 39 Similarly, an image 4001 (e.g., image B) is displayed on the screen 2005 of the portable terminal 2000, and based on this image B, the same image 4002 (e.g., image B) is also displayed on the screen of the spatial levitation image display device 1000, i.e., the spatial levitation image 3. Figure 29 (Dual display function 2910). One technical problem is that, in such a state, if the user 230, for example, zooms on image B of screen 2005 of portable terminal 2000 via touch operation (especially pinch operation), how the two images 4001 and 4002 will change is unclear in the past.

[0535] In this embodiment, one solution is as follows. The system detects a user's touch operation (e.g., an outward pinch operation) on image 4001 (image B) of the screen 2005 of the portable terminal 2000, and magnifies the image 4001 corresponding to the outward pinch operation. Image B becomes the magnified image Bd. In response to this detection, the system synchronizes the magnified image display to reflect it on image 4002 (image B) on one side of the spatially suspended image 3. Figure 29 Synchronization control function 2920). For example Figure 40 As shown in the lower part, the same applies when shrinking the display by pinching inwards.

[0536] The synchronization control described in this example is based on the setting that makes touch operation detection and zooming display valid on the portable terminal 2000 side, and makes touch operation detection and zooming display invalid on the spatial floating image 3 side. Figure 29 The system implements operation detection and control functions (2930). When two devices are connected in communication, the system sets the post-connection state. Each device operates, detects, and displays according to these post-connection settings. This achieves the following: Figure 40 The effects shown are as follows.

[0537] As described in the examples above, there are cases where a portable terminal 2000 is connected to a spatial levitation image display device 1000, and a target image / video is displayed on both the screen of the portable terminal 2000 and the screen of the spatial levitation image display device 1000 (spatial levitation image 3) (dual display). In this case, there are cases where the user 230 performs touch operations on the displayed image on the portable terminal 2000 side and on the displayed image on the spatial levitation image 3 side. Whether the user operation performed on one device should be reflected on the displayed image on the device where the operation was not performed has been unclear in the past, and various possible solutions can be considered. In the state after communication connection, especially in the state of dual display, the hardware and software operation and detection of one or both of the devices—the operation and detection on the portable terminal 2000 side and the operation and detection on the spatial levitation image 3 side—need to be appropriately set and controlled. By appropriately setting and controlling the display images on the portable terminal 2000 side and the spatial levitation image 3 side according to the user 230's requirements, clear user operations and corresponding responses (display control processing) can be achieved. For example, by setting the portable terminal 2000 side to active and the spatial levitation image 3 side to inactive, for example in... Figure 39 When user 230 rotates the display image on one side of portable terminal 2000, the display image on one side of spatial floating image 3 can be rotated synchronously with this operation. This clarifies the user's operation and improves the convenience for user 230.

[0538] As described above, after the communication connection is established, operation detection is only enabled on the portable terminal 2000 side during a certain time period and mode, while operation detection is only enabled on the spatial levitation image display device 1000 side during other time periods and modes. The specified functions can be achieved in each mode. When operation detection is enabled on one device, useless operation detection on the other device can be prevented. Depending on the specified mode, operation detection on both devices can be enabled.

[0539] Furthermore, when the portable terminal 2000 and the spatial levitation image display device 1000 lose their communication connection, this system restores the operation detection settings for each device as described above to the settings before the connection was established, or the normal settings.

[0540] [Spatial Suspended Image Display System and Display Method]

[0541] Figure 29 This is an explanatory diagram of the spatial levitation image display system and display method of Embodiment 5. Figure 29System 2900 is a spatial levitation image display system formed by the communication connection between spatial levitation image display device 1000 and portable terminal 2000. Using Figure 29 The summary of Example 5 is described below.

[0542] State A indicates a situation where, after the spatial levitation image display device 1000 is communicatively connected to the portable terminal 2000, for example, the image 2901 on the screen 2005 of the portable terminal 2000 is also displayed as image 2902 in the spatial levitation image 3 (display range 3R) of the spatial levitation image display device 1000. State A is a dual display state of image 2901 on the portable terminal 2000 side and image 2902 on the spatial levitation image 3 side. This dual display is a state based on the same image data being displayed as images 2901 and 2902 on the screens of each device in different ways.

[0543] State B indicates, for example, a situation where user 230 performs a touch operation on image 2901 of screen 2005 of portable terminal 2000 in state A. Figure 29 In this example, the post-connection settings (a mode described later) in this system are set to enable operation detection on the portable terminal 2000 side and disable operation detection on the spatial levitation image 3 side. Image 2901 is the object subjected to touch operation, and pre-defined processing associated with image 2901 is performed, such as display update. In this example, because the touch operation and detection of image 2901 are determined to be valid, image 2901 becomes image 2903 due to display update.

[0544] Next, based on the detected touch operation and display update of image 2901 on the portable terminal 2000 side, this system collaborates with the spatial levitation image display device 1000 side via communication. As a synchronization control, image 2903 on the spatial levitation image 3 side also reflects the processing corresponding to the touch operation. That is, as a result of the touch operation, image 2902 on the spatial levitation image 3 side is also updated. Image 2902 becomes image 2904 due to the display update. In this way, when the portable terminal 2000 side is set to active, if user 230 performs a user operation on the portable terminal 2000 side, the spatial levitation image 3 side can also reflect the user operation.

[0545] On the other hand, if user 230 performs a touch operation on image 2902 displayed on one side of the spatial levitation image 3, because the operation detection on the side of the spatial levitation image 3 is set to invalid, this system does not reflect the touch operation on image 2902 on the side of the spatial levitation image 3 and image 2901 on the side of the portable terminal 2000. As a result, neither side updates its display, thus maintaining the state of image 2902 on the side of the spatial levitation image 3 and image 2901 on the side of the portable terminal 2000.

[0546] exist Figure 29 In system 2900, as functions for setting and controlling as described in the above examples, there are dual display function 2910, synchronous control function 2920 and operation detection control function 2930.

[0547] The dual display function 2910 is a function in which, in response to a communication connection, the image / video displayed on the screen of the spatial levitation image 3 of the spatial levitation image display device 1000 is displayed on the screen of the portable terminal 2000, and / or the image / video displayed on the screen of the portable terminal 2000 is displayed on the screen (screen) of the spatial levitation image 3 of the spatial levitation image display device 1000. The synchronization control function 2920 is a function in which, when the user 230 performs an operation such as a touch operation on the screen of one device, a predetermined process (in other words, display control processing, system processing) corresponding to that operation is synchronously reflected on the display image of the other device's screen via communication. Furthermore, the synchronization control function 2920 can be integrated into the dual display function 2910.

[0548] Furthermore, the operation detection and control function 2930 is a function that sets and controls which side or both of the operation / detection / display control on the spatial levitation image display device 1000 side and the operation / detection / display control on the portable terminal 2000 side are effective / ineffective regarding user touch operations and other operations performed on the screens of each device and the prescribed processing performed accordingly.

[0549] [Dual display function]

[0550] Figure 30 It is about Figure 29The diagram illustrates the dual display function 2910. As a first example of dual display, state A shows the case where image A displayed in the spatial levitation image 3 of the spatial levitation image display device 1000 is also displayed as image A on the screen 2005 of the portable terminal 2000. Initially, image A is displayed as image 3002 in the spatial levitation image 3 of the spatial levitation image display device 1000. User 230 establishes a communication connection between the portable terminal 2000 and the spatial levitation image display device 1000. The spatial levitation image display device 1000 sends image data corresponding to image A to the portable terminal 2000 via communication, and the portable terminal 2000 obtains the image data corresponding to image A from the spatial levitation image display device 1000 via communication, displaying image A as image 3001 on the screen 2005. User 230 can view image A in the spatial levitation image 3 and can also view the corresponding image A on the screen 2005 of the portable terminal 2000.

[0551] As a second example of dual display, state B shows a case where image B displayed on the screen 2005 of the portable terminal 2000 is also displayed as image B in the spatial levitation image 3 of the spatial levitation image display device 1000. Initially, image B is displayed as image 3003 on the screen 2005 of the portable terminal 2000. User 230 establishes a communication connection between the portable terminal 2000 and the spatial levitation image display device 1000. The portable terminal 2000 sends image data corresponding to image B to the spatial levitation image display device 1000 via communication, and the spatial levitation image display device 1000 obtains the image data corresponding to image B from the portable terminal 2000 and displays image B as image 3004 in the spatial levitation image 3. User 230 can view image B on the screen 2005 of the portable terminal 2000 and can also view the corresponding image B in the spatial levitation image 3.

[0552] [A variation of image data acquisition]

[0553] like Figure 30 and the following Figure 32BAs shown, target image data can be transmitted in the direction from the spatial levitation image display device 1000 to the portable terminal 2000, or in the opposite direction, but is not limited to this. In a variation, for example, the spatial levitation image display device 1000 transmits ID information of the target image to the portable terminal 2000. The portable terminal 2000 determines the target image data stored in its memory based on the received ID information, and displays the target image on the screen 2005 based on the determined image data. Alternatively, the portable terminal 2000, based on the received ID information (which could be a URL, etc.), refers to an external server, determines the target image data stored on that external server, retrieves the determined image data from the external server, and displays the target image on the screen 2005 based on the image data.

[0554] [Basic Control Flow]

[0555] Figure 31 This refers to the spatial levitation image display system of Example 5 ( Figure 29 The basic control flow in the system 2900 and display method. The control processing in this embodiment is performed by the main body of each device of the spatial levitation image display device 1000 and the portable terminal 2000, especially the control unit of the portable terminal 2000 and the image processing unit of the spatial levitation image display device 1000. Unless otherwise stated below, the main body is the same.

[0556] Step S10 represents the state before the spatial levitation image display device 1000 and the portable terminal 2000 establish a communication connection (in other words, the unconnected state). Each device can be used arbitrarily. Furthermore, in step S10, the preset system settings and user settings can be the mode selection / setting described later.

[0557] From the pre-connection state in step S10, in step S11, the portable terminal 2000 is communicatively connected to the spatial levitation image display device 1000. For example, based on the operation and actions of the user 230 holding the portable terminal 2000, the portable terminal 2000 is communicatively connected to the spatial levitation image display device 1000. This communication connection is any connection method independent of details such as the communication interface. This communication connection can be established by the portable terminal 2000 sending a connection request to the spatial levitation image display device 1000, as in conventional methods. This communication connection can also be a method specific to embodiments 3 and 4 described above, i.e., a communication connection established by bringing the portable terminal 2000 into contact with the spatial levitation image 3.

[0558] Step S12 deals with the processing and status at the point immediately after establishing a communication connection. In step S12, settings are configured for each device in this system—the portable terminal 2000 and the spatial levitation image display device 1000—during connection. In other words, settings are configured immediately after connection, specifying the system's post-connection status. Each device performs its own "post-connection settings" based on communication cooperation with the other device. These settings include configurations related to hardware, software, and functions. These settings primarily concern… Figure 29 The operation detection function 2930 configures the validity / invalidity settings for user operations and sensor detection in each device. This setting can be configured for... Figure 29 The dual display function 2910 and synchronous control function 2920 include settings for the images displayed on the screens of each device. Details of the "post-connection settings" are described later. Figure 32B ).

[0559] As part of the operation detection settings on the portable terminal 2000 side, these settings include settings for whether the sensors for detecting touch operations and rotation operations performed on the screen of the portable terminal 2000 are active or inactive. Additionally, as part of the operation detection settings on the spatial levitation image display device 1000 side, these settings include settings for whether the sensors for detecting touch operations and rotation operations performed on the screen (display area 3R) of the spatial levitation image 3 are active or inactive. The "post-connection settings" remain in effect until the communication connection is terminated as described later.

[0560] In step S12, the system can select and set the mode described later. For example, a GUI for selecting and setting the mode can be displayed on the screen of a device, and the user 230 can select and set the mode.

[0561] Step S13 describes the user 230's use of the various devices in the system under the post-connection settings (in other words, the communication connection state). For example, as a dual display, the image displayed on one side of the spatial levitation image 3 is also displayed on the screen of the portable terminal 2000. Alternatively, the image on the portable terminal 2000 is also displayed on the spatial levitation image 3. The user 230 uses the image displayed on the screen of the portable terminal 2000 and the image displayed in the spatial levitation image 3 of the spatial levitation image display device 1000. At this time, user operations such as touch operations and rotation operations performed by the user on the display images of each device's screen are controlled for validity / invalidation, synchronization, etc., based on the post-connection settings in step S12. For example, in a certain setting mode, touch operations performed on the display image on the portable terminal 2000 side are valid, and in response to this operation, the display images of each device are updated, etc., but touch operations performed on the display image on the spatial levitation image 3 side are invalid. Thus, the operation and detection during dual display become clear, and the user 230 can operate more easily. Details of each mode are described later.

[0562] In step S14, the communication connection between the portable terminal 2000 and the spatial levitation image display device 1000 is terminated. For example, based on the operation or action of the user 230 holding the portable terminal 2000, the portable terminal 2000 terminates the communication connection with the spatial levitation image display device 1000. The termination of connection includes not only cases where the user 230 intentionally disconnects or terminates the connection, but also cases where the connection is accidentally terminated due to errors or other reasons. The system becomes disconnected in step S14, in other words, in an unconnected state, and reverts to the state before connection.

[0563] Following step S14, in step S15, settings are configured when the portable terminal 2000 and the spatial levitation image display device 1000 of this system disconnect from each other. In other words, settings are configured immediately after disconnection, regarding the pre-connection state of this system, and pre-connection settings. These settings include settings regarding the hardware, software, and functions of each device. Each device performs its own "pre-connection settings." This configuration is equivalent to restoring to the pre-connection state of step S10. For example, restoring to the state described later. Figure 32A The system can store the "pre-connection settings" in memory during the communication connection in steps S11 and S12, and restore the "pre-connection settings" based on the information in memory when the connection is terminated in steps S14 and S15.

[0564] [Examples of display and settings before and after connection]

[0565] Figure 32A and Figure 32BThis is a diagram that summarizes examples (in other words, modes or patterns) of the display and settings before and after the portable terminal 2000 and the spatial levitation image display device 1000 are connected in the communication form in this system.

[0566] Figure 32A This shows an example of the display / settings before connection. The settings correspond to... Figure 31 The state of step S10. As a pre-connection setting for this system, the table shows the settings of the portable terminal 2000 (hereinafter referred to as MT) and the spatial levitation image display device 1000 (hereinafter referred to as AD). The spatial levitation image display device 1000 displays a predetermined image on the screen (screen) of the spatial levitation image 3, such as image A selected by user 230 (in other words, content A). The portab...

Claims

1. An aerial levitation image display device, characterized in that, include: Image Processing Department; The display unit displays the image processed by the image processing unit. An optical system that generates an aerial levitation image based on the image displayed on the display unit; The detection unit detects the contact of the object with the display area of ​​the suspended aerial image; and The communications department communicates with users' portable terminals. Specifically, when the portable terminal is detected to be in contact with the display area, a target image specified on one side of the portable terminal is displayed as the aerial levitation image.

2. The aerial levitation image display device as described in claim 1, characterized in that: Before, during, or after contact between the portable terminal and the display area, a display request is sent from the portable terminal to the aerial levitation image display device. When the portable terminal is detected to be in contact with the display area, the display request is accepted, and the target image specified on one side of the portable terminal is obtained from the portable terminal or an external device via communication and displayed as the aerial levitation image.

3. The aerial levitation image display device as described in claim 1, characterized in that: If the portable terminal is detected to be in contact with a specific area within the display range, the target image specified on one side of the portable terminal will be displayed as the suspended image in the air.

4. The aerial levitation image display device as described in claim 1, characterized in that: When a specific posture of the portable terminal is detected when it comes into contact with the display area, a target image specified on one side of the portable terminal is displayed as the aerial levitation image.

5. The aerial levitation image display device as described in claim 1, characterized in that: If, after the portable terminal detects contact with the display area, it enters a space deeper than the display area, the target image specified on one side of the portable terminal will be displayed as the floating image in the air.

6. The aerial levitation image display device as described in claim 1, characterized in that: When the portable terminal comes into contact with the display area, if the contact time reaches a certain period of time, the target image specified on one side of the portable terminal will be displayed as the floating image in the air.

7. The aerial levitation image display device as described in claim 1, characterized in that: The guiding image used to remind the user to perform the operation of bringing the portable terminal into contact with the display area is displayed as the spatial levitation image.

8. The aerial levitation image display device as described in claim 1, characterized in that: A guide image is displayed on the screen of the portable terminal to remind the user to perform the operation of bringing the portable terminal into contact with the display area.

9. The aerial levitation image display device as described in claim 1, characterized in that: If an invisible area is detected due to the contact between the portable terminal and the display area, a non-display area containing the invisible area is determined in the display area, and the spatial levitation image is displayed in the display area outside the non-display area.

10. The aerial levitation image display device as described in claim 1, characterized in that: The detection unit is at least one of an air operation detection sensor and a camera unit, wherein the air operation detection sensor is used to detect air operations performed on a range including the display range, and the camera unit captures images of the range including the display range.

11. The aerial levitation image display device as described in claim 1, characterized in that: Information is obtained by identifying the barcode displayed on the screen of the portable terminal. When the barcode is recognized and the portable terminal is detected to be in contact with the display area, the target image specified on one side of the portable terminal is displayed as the aerial levitation image.

12. The aerial levitation image display device as described in claim 1, characterized in that: The portable terminal identifies the barcode displayed in the spatial levitation image to obtain information. When the barcode is recognized and the portable terminal is detected to be in contact with the display area, the target image specified on one side of the portable terminal is displayed as the aerial levitation image.

13. The aerial levitation image display device as described in claim 11 or 12, characterized in that: The information in the barcode includes at least one of the following: The portable terminal sends a display request to the aerial levitation image display device, the communication connection information between the portable terminal and the aerial levitation image display device, the ID of the portable terminal or the ID of the user, and the ID or URL of the target image.

14. The aerial levitation image display device as described in claim 1, characterized in that: When the portable terminal is detected to be in contact with the display area, the portable terminal acquires the image being displayed in the spatial levitation image.

15. An aerial levitation image display system, comprising an aerial levitation image display device and a user's portable terminal, characterized in that: The aerial levitation image display device includes: Image Processing Department; The display unit displays the image processed by the image processing unit. An optical system that generates an aerial levitation image based on the image displayed on the display unit; The detection unit detects the contact of the object with the display area of ​​the suspended aerial image; and The communications department communicates with users' portable terminals. When the aerial levitation image display device detects that the portable terminal is in contact with the display area, it displays a target image specified on one side of the portable terminal as the aerial levitation image.

16. A portable terminal, which is the portable terminal in an aerial levitation image display system comprising an aerial levitation image display device and a user's portable terminal, characterized in that: The aerial levitation image display device includes: Image Processing Department; The display unit displays the image processed by the image processing unit. An optical system that generates an aerial levitation image based on the image displayed on the display unit; The detection unit detects the contact of the object with the display area of ​​the suspended aerial image; and The communication unit communicates with the user's portable terminal. Specifically, when the aerial levitation image display device detects that the portable terminal is in contact with the display area, it displays a target image designated on one side of the portable terminal as the aerial levitation image. The portable terminal specifies the target image based on the user's operation and sends the image data or information of the specified target image to the aerial floating image display device.

17. An aerial levitation image display device, characterized in that, include: The image processing department performs image processing. The display unit displays the image processed by the image processing unit. An optical system that generates an aerial levitation image based on the image displayed on the display unit; The user operation detection mechanism detects the user's operations on the display range of the aerial suspended image; and The communication unit communicates with the user's portable terminal. Specifically, based on the communication connection with the user's portable terminal, the first image displayed in the aerial levitation image is also displayed as the first image on the screen of the portable terminal, and / or the second image displayed on the screen of the portable terminal is also displayed as the second image in the aerial levitation image. Based on the detection and judgment of the user's operation on the first image displayed in the aerial levitation image, a predetermined first process associated with the first image of the aerial levitation image is executed, and control is performed to reflect the first process on the first image on the screen of the portable terminal. Alternatively, based on the detection and judgment of the user's operation on the second image displayed on the screen of the portable terminal, a predetermined second process associated with the second image on the screen of the portable terminal is executed, and control is performed to reflect the second process on the second image of the aerial levitation image.

18. The aerial levitation image display device as described in claim 17, characterized in that: As a setting following the communication connection, the operation and detection of the user's operation on the displayed image of the aerial levitation image on the aerial levitation image display device side, and the operation and detection of the user's operation on the displayed image on the screen on the portable terminal side are set to be valid / invalid, and the operation, detection, display and processing are controlled according to the setting.

19. The aerial levitation image display device as described in claim 18, characterized in that: When the communication connection is established, the validity / invalidation setting is performed after the communication connection is established. When the communication connection is terminated, the valid / invalid setting is restored to the setting before the communication connection was established.

20. The aerial levitation image display device as described in claim 18, characterized in that: The user's operations on the suspended aerial image or the image displayed on the screen of the portable terminal include touch operations and rotation operations. For the touch operation, the touch sensor in the user operation detection mechanism of the aerial levitation image display device, or the touch sensor of the portable terminal, is used for detection. The rotation operation is detected using the attitude sensor of the aerial levitation image display device or the attitude sensor of the portable terminal.

21. The aerial levitation image display device as described in claim 18, characterized in that: In one mode, the user's operation on the displayed image of the aerial floating image on the aerial floating image display device side is set to be valid, while the user's operation on the displayed image on the screen on the portable terminal side is set to be invalid.

22. The aerial levitation image display device as described in claim 18, characterized in that: In one mode, the user's operation on the displayed image of the aerial floating image on the aerial floating image display device side is set to invalid, while the user's operation on the displayed image on the screen on the portable terminal side is set to valid.

23. The aerial levitation image display device as described in claim 18, characterized in that: In one mode, the user's operation on the displayed image of the aerial floating image on the aerial floating image display device side and its detection are set to be valid, and the user's operation on the displayed image on the screen on the portable terminal side and its detection are also set to be valid.

24. The aerial levitation image display device as described in claim 18, characterized in that: Mode options can be displayed in the aerial levitation image of the aerial levitation image display device and / or on the screen of the portable terminal. The mode options correspond to the setting of whether the user on the aerial levitation image display device performs an operation on the displayed image of the aerial levitation image and the detection thereof, and whether the user on the portable terminal performs an operation on the displayed image on the screen and the detection thereof is valid or invalid. The validity / invalidity is set according to the user's selection of the mode.

25. The aerial levitation image display device as described in claim 20, characterized in that: The touch sensor in the aerial levitation image display device includes an aerial operation detection sensor or camera that detects the position of the user's finger or the object being manipulated relative to the display area of ​​the aerial levitation image.

26. The aerial levitation image display device as described in claim 20, characterized in that: The rotation operation includes rotating the housing of the aerial levitation image display device or the housing of the portable terminal. The prescribed processing corresponding to the determination of the rotation operation includes displaying the image in the aerial levitation image or the image on the screen of the portable terminal in a state where rotation has occurred from the user's perspective.

27. The aerial levitation image display device as described in claim 17, characterized in that: When the communication connection is established, based on the detection that the portable terminal comes into contact with the display range of the aerial levitation image, the first image displayed in the aerial levitation image is also displayed as the first image on the screen of the portable terminal, and / or the second image displayed on the screen of the portable terminal is also displayed as the second image in the aerial levitation image.

28. A display method in a system comprising an aerial levitating image display device and a user's portable terminal, characterized in that: The aerial levitation image display device includes: The image processing department performs image processing. The display unit displays the image processed by the image processing unit. An optical system that generates an aerial levitation image based on the image displayed on the display unit; The user operation detection mechanism detects the user's operations on the display area of ​​the aerial levitation image; and The communication unit communicates with the user's portable terminal. The display method includes: The first step is to establish a communication connection between the aerial levitation image display device and the user's portable terminal; The second step involves, based on the communication connection, displaying the first image shown in the aerial levitation image as the first image on the screen of the portable terminal, and / or displaying the second image shown on the screen of the portable terminal as the second image in the aerial levitation image; and The third step involves detecting and determining the user's operation on the first image displayed in the aerial levitation image, executing a predetermined first process associated with the first image of the aerial levitation image, and controlling the first process to also reflect the first process on the first image on the screen of the portable terminal, and / or detecting and determining the user's operation on the second image displayed on the screen of the portable terminal, executing a predetermined second process associated with the second image on the screen of the portable terminal, and controlling the second process to also reflect the second process on the second image of the aerial levitation image.

29. A portable terminal in a system comprising an aerial levitation image display device and a user's portable terminal, characterized in that: The aerial levitation image display device includes: The image processing department performs image processing. The display unit displays the image processed by the image processing unit. An optical system that generates an aerial levitation image based on the image displayed on the display unit; The user operation detection mechanism detects the user's operations on the display area of ​​the aerial levitation image; and The communication unit communicates with the user's portable terminal. The portable terminal performs the following control: Establish a communication connection with the aerial levitation image display device; Based on the communication connection, the first image displayed in the aerial levitation image is also displayed as the first image on the screen of the portable terminal, and / or the second image displayed on the screen of the portable terminal is also displayed as the second image in the aerial levitation image; and Based on the detection and judgment of the user's operation on the first image displayed in the aerial levitation image, a predetermined first process associated with the first image of the aerial levitation image is executed, and control is performed to reflect the first process on the first image on the screen of the portable terminal. Alternatively, based on the detection and judgment of the user's operation on the second image displayed on the screen of the portable terminal, a predetermined second process associated with the second image on the screen of the portable terminal is executed, and control is performed to reflect the second process on the second image of the aerial levitation image.

Images