Aerial suspension image display device and character display device

Abstract

The invention provides a better aerial suspension image display device. According to the invention, it is possible to contribute to "3 good health and welfare", "9 industry, innovation and infrastructure", "11 sustainable cities and communities" of sustainable development objectives (SDGs). In an aerial floating image display device, an image processing unit performs image processing on an image generated by rendering a virtual 3D space of an object in which a character is disposed, the rendering of the virtual 3D space being performed by photographing by a perspective projection method by a virtual 3D space camera set in the virtual 3D space. The image generated by rendering and displayed on the display unit is an image obtained by setting the field angle of the virtual 3D space camera so as to satisfy Lf > = Lm * Fi / Pa, imaging the virtual 3D space in which the object of the character is disposed, and rendering the image.

Description

TECHNICAL FIELD

[0001] The present application relates to an aerial floating image display device. BACKGROUND

[0002] As for the aerial floating image display technology, for example, there is a disclosure in Patent Literature 1.

[0003] PRIOR ART DOCUMENTS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2019-128722 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] However, in the disclosure of Patent Literature 1, there is no sufficient consideration on how to obtain a practical brightness and quality of the aerial floating image, and how to make the user more enjoyable to watch the aerial floating image, and the like.

[0008] An object of the present application is to provide a better aerial floating image display device.

[0009] MEANS OF SOLVING THE PROBLEM

[0010] In order to solve the above-mentioned problems, for example, the structure described in the claimed technical solution is adopted. The present application includes a plurality of technical solutions to solve the above-mentioned problems. For example, one of the technical solutions is an aerial floating image display device, which can be configured to include: an image processing unit; a display unit that displays an image processed by the image processing unit; and an optical system that generates an aerial floating image based on the image displayed by the display unit, wherein the image processing unit performs image processing on an image generated by rendering a virtual 3D space in which an object on which a character is arranged, the rendering of the virtual 3D space being performed by a virtual 3D space camera provided in the virtual 3D space by perspective projection, the image generated by the rendering and displayed on the display unit being an image obtained by rendering the virtual 3D space in which the object on which the character is arranged by shooting the virtual 3D space with the virtual 3D space camera, the field of view of the virtual 3D space camera being set in such a way as to satisfy Lf≥Lm×Fi / Pa, where Lf is a focal length of a lens of the virtual 3D space camera converted to a 35mm film, Lm is a distance at which a user of the aerial floating image display device visually views a display screen of the aerial floating image, Fi is a diagonal length of the 35mm film, and Pa is a diagonal length of the display screen of the aerial floating image.

[0011] EFFECT OF THE INVENTION

[0012] According to the present application, a better aerial floating image display device can be realized. In addition to this, the technical problems, technical features and technical effects will be clear through the following embodiment description. BRIEF DESCRIPTION OF DRAWINGS

[0013] Figure 1 is a drawing showing an example of a use form of the spatial floating image display device of one embodiment of the present application.

[0014] Figure 2A is a drawing showing an example of a main part structure and a retroreflective part structure of the spatial floating image display device of one embodiment of the present application.

[0015] Figure 2B is a drawing showing an example of a main part structure and a retroreflective part structure of the spatial floating image display device of one embodiment of the present application.

[0016] Figure 2C is a drawing showing an example of a main part structure and a retroreflective part structure of the spatial floating image display device of one embodiment of the present application.

[0017] Figure 2D is a drawing showing an example of a main part structure and a retroreflective part structure of the aerial floating image display device of one embodiment of the present application.

[0018] Figure 2E is a projection view of the retroreflective plate constituting the aerial floating image display device in one embodiment of the present application.

[0019] Figure 2F is a plan view of the retroreflective plate constituting the aerial floating image display device in one embodiment of the present application.

[0020] Figure 2G is a perspective view showing the corner reflector included in the retroreflective plate constituting the aerial floating image display device in one embodiment of the present application.

[0021] Figure 2H is a plan view showing the corner reflector included in the retroreflective plate constituting the aerial floating image display device in one embodiment of the present application.

[0022] Figure 2I is a side view showing the corner reflector included in the retroreflective plate constituting the aerial floating image display device in one embodiment of the present application.

[0023] Figure 3 is a drawing showing a structure example of the spatial floating image display device of one embodiment of the present application.

[0024] Figure 4AFIG. 1 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0025] Figure 4B FIG. 2 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0026] Figure 4C FIG. 3 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0027] Figure 4D FIG. 4 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0028] Figure 4E FIG. 5 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0029] Figure 4F FIG. 6 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0030] Figure 4G FIG. 7 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0031] Figure 4H FIG. 8 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0032] Figure 4I FIG. 9 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0033] Figure 4J FIG. 10 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0034] Figure 4K FIG. 11 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0035] Figure 4L FIG. 12 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0036] Figure 4M FIG. 13 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0037] Figure 4N FIG. 14 is a diagram showing an example of a structure of a spatial suspended image display device according to an embodiment of the present application.

[0038] Figure 4OFIG. 1 is a diagram showing an example of the structure of a spatially suspended image display device according to an embodiment of the present application.

[0039] Figure 5 FIG. 2 is a cross-sectional view showing an example of the detailed structure of a light source device according to an embodiment of the present application.

[0040] Figure 6 FIG. 3 is a cross-sectional view showing an example of the detailed structure of a light source device according to an embodiment of the present application.

[0041] Figure 7 FIG. 4 is a cross-sectional view showing an example of the detailed structure of a light source device according to an embodiment of the present application.

[0042] Figure 8 FIG. 5 is a diagram showing an example of the arrangement of main parts of a spatially suspended image display device according to an embodiment of the present application.

[0043] Figure 9 FIG. 6 is a cross-sectional view showing an example of the structure of a display device according to an embodiment of the present application.

[0044] Figure 10 FIG. 7 is a cross-sectional view showing an example of the structure of a display device according to an embodiment of the present application.

[0045] Figure 11 FIG. 8 is a diagram for explaining the light source diffusion characteristics of an image display device according to an embodiment of the present application.

[0046] Figure 12 FIG. 9 is a diagram for explaining the diffusion characteristics of an image display device according to an embodiment of the present application.

[0047] Figure 13A FIG. 10 is a diagram showing an example of the technical problem to be solved by image processing according to an embodiment of the present application.

[0048] Figure 13B FIG. 11 is a diagram showing an example of image processing according to an embodiment of the present application.

[0049] Figure 13C FIG. 12 is a diagram showing an example of image display processing according to an embodiment of the present application.

[0050] Figure 13D FIG. 13 is a diagram showing an example of image display processing according to an embodiment of the present application.

[0051] Figure 14 FIG. 14 is a diagram showing an example of the main part structure and the retroreflective portion structure of a spatially suspended image display device according to an embodiment of the present application.

[0052] Figure 15A FIG. 15 is a diagram showing an example of a display example of a spatially suspended image display device according to an embodiment of the present application.

[0053] Figure 15B FIG. 1 is a diagram illustrating an example of a display of a spatial floating image display device according to an embodiment of the present invention.

[0054] Figure 15C FIG. 2 is a diagram illustrating an example of a display of a spatial floating image display device.

[0055] Figure 15D FIG. 3 is a diagram illustrating an example of user perception of a display of a spatial floating image display device.

[0056] Figure 16A FIG. 4 is a diagram illustrating an example of a method of generating a rendered image of a 3D model of a character on a virtual 3D space.

[0057] Figure 16B FIG. 5 is a diagram illustrating an example of a rendering image display process of a spatial floating image display device according to an embodiment of the present invention.

[0058] Figure 16C FIG. 6 is a diagram illustrating an example of a user visual distance of a spatial floating image display device according to an embodiment of the present invention.

[0059] Figure 16D FIG. 7 is a diagram illustrating an example of an arm length calculation result based on a survey result.

[0060] Figure 16E FIG. 8 is a diagram illustrating an example of a focal length of a virtual 3D space camera used in rendering according to an embodiment of the present invention.

[0061] Figure 16F FIG. 9 is a diagram illustrating an example of a focal length of a virtual 3D space camera used in rendering according to an embodiment of the present invention.

[0062] Figure 17A FIG. 10 is a diagram illustrating an example of a display of a spatial floating image display device according to an embodiment of the present invention.

[0063] Figure 17B FIG. 11 is a diagram illustrating an example of user perception of a display of a spatial floating image display device according to an embodiment of the present invention.

[0064] Figure 18 FIG. 12 is a diagram illustrating an example of a specified area in a spatial floating image of a spatial floating image display device according to an embodiment of the present invention.

[0065] Figure 19A FIG. 13 is a diagram illustrating an example of a character session device and a character session system according to an embodiment of the present invention.

[0066] Figure 19BFIG. 1 is a diagram illustrating an example of the action of a role conversation device and a role conversation system according to an embodiment of the present application.

[0067] Figure 19C FIG. 2 is a diagram illustrating an example of the action of a role conversation device and a role conversation system according to an embodiment of the present application.

[0068] Figure 19D FIG. 3 is a diagram illustrating an example of the action of a role conversation device and a role conversation system according to an embodiment of the present application.

[0069] Figure 19E FIG. 4 is a diagram illustrating an example of the action of a role conversation device and a role conversation system according to an embodiment of the present application.

[0070] Figure 19F FIG. 5 is a diagram illustrating an example of the action of a role conversation device and a role conversation system according to an embodiment of the present application.

[0071] Figure 19G FIG. 6 is a diagram illustrating an example of the action of a role conversation device and a role conversation system according to an embodiment of the present application.

[0072] Figure 19H FIG. 7 is a diagram illustrating an example of the action of a role conversation device and a role conversation system according to an embodiment of the present application.

[0073] Figure 19I FIG. 8 is a diagram illustrating an example of the action of a role conversation device and a role conversation system according to an embodiment of the present application.

[0074] Figure 19J FIG. 9 is a diagram illustrating an example of the action of a role conversation device and a role conversation system according to an embodiment of the present application. DETAILED DESCRIPTION

[0075] Embodiments of the present application will be described in detail below based on the accompanying drawings. However, the present application is not limited to the description of the embodiments, and those skilled in the art can implement various modifications and changes within the scope of the technical idea disclosed in the present specification. In addition, the same reference numerals are used to designate the same parts having the same functions in all the drawings used to describe the present application, and the repeated description thereof will be omitted.

[0076] The following embodiment relates to an image display device capable of making an image formed by image light from an image light source be transmitted through a transparent member for partitioning a space such as glass, and displaying the image as a space floating image outside the transparent member. In addition, in the following embodiment, the term "space floating image" is used to express an image floating in a space. Instead of this term, the terms "airborne image", "space image", "air floating image", "displayed image space floating optical image", "displayed image airborne optical image", and the like can be used. The term "space floating image" used mainly in the description of the embodiment is used as a representative example of these terms.

[0077] According to the following embodiment, a good image display device can be implemented, for example, in an ATM of a bank, a ticket vending machine of a station, a digital signboard, and the like. For example, a touch panel is generally used in the ATM of a bank, the ticket vending machine of a station, and the like at present, but a transparent glass surface or a light-transmissive plate on which high-resolution image information is displayed in a space floating state can be used. At this time, by making the divergence angle of the outgoing image light become a sharp angle, and further making it uniform to a specific polarization, only the normally reflected light with respect to the retroreflective plate can be efficiently reflected, so that the light utilization efficiency is high, and a ghost image other than the main space floating image, which is a problem in the existing retroreflective method, can be suppressed, and a clear space floating image can be obtained. In addition, by the device including the light source of the embodiment, a novel and usable space floating image display device (space floating image display system) capable of greatly reducing power consumption can be provided. In addition, for example, in a vehicle, a space floating image display device for a vehicle capable of performing so-called one-way space floating image display, which can be viewed in the interior and / or the exterior of the vehicle, can be provided.

[0078] <Embodiment 1>

[0079] <Example of usage form of space floating image display device>

[0080] Figure 1This figure illustrates an example of a spatial floating image display device according to one embodiment of the present invention, and shows the overall structure of the spatial floating image display device. The specific structure of the spatial floating image display device will be described in detail using Figure 2 and other figures. Light with a narrow-angle directivity and specific polarization, emitted from the image display device 1, serves as an image beam. After reflection and other processes within the optical system of the spatial floating image display device, it first enters the retroreflective plate 2. After retroreflection, it passes through a transparent member 100 (such as glass), forming a real aerial image (the spatial floating image 3) on the outer surface of the glass. In the following embodiments, the retroreflective plate 2 (retroreflective plate) is used as an example of a retroreflective member. However, the retroreflective plate 2 of the present invention is not limited to a planar plate; it is used for illustrative purposes only and encompasses sheet-like retroreflective members that can be attached to planar or non-planar surfaces, as well as an entire assembly comprising a sheet-like retroreflective member attached to a planar or non-planar surface. Furthermore, because the light reflected from the retroreflective plate 2 has optical properties capable of forming an image, the retroreflective plate 2 can also be referred to as an imaging optical member or imaging optical plate.

[0081] Furthermore, in stores and the like, display windows (also referred to as "windowpanes") 105, formed of a light-transmitting member such as glass, serve to partition the space. The spatial floating image display device of this embodiment can display a floating image unidirectionally, either to the exterior or the interior of the store (space), through this transparent member.

[0082] Figure 1 In the figure, the inside of the window glass 105 (inside the store) is represented as the depth direction (depth direction), and the outside (for example, the sidewalk) is represented as the near side. Alternatively, by providing a mechanism for reflecting light with a specific polarization on the window glass 105, an aerial image can be formed at a desired position within the store.

[0083] <Configuration Example of the Optical System of a Spatial Floating Image Display Device>

[0084] Figure 2A This is a diagram showing an example of the structure of an optical system of a spatial floating image display device according to an embodiment of the present invention. Figure 2A The structure of the space suspended image display device is described in more detail. Figure 2A As shown in (1), a display device 1 is provided that diffuses image light of specific polarization at a narrow angle in an oblique direction of a transparent member such as glass 100. The display device 1 includes a liquid crystal display panel 11 and a light source device 13 that generates light of specific polarization having a narrow-angle diffusion characteristic.

[0085] Image light of a specific polarization from the display device 1 is reflected by a polarization separation component 101 (in the figure, the polarization separation component 101 is formed into a sheet and attached to the transparent component 100) provided on the transparent component 100 and has a film that selectively reflects image light of a specific polarization, and is incident on the retroreflective plate 2. A λ / 4 wave plate 21 is provided on the image light incident surface of the retroreflective plate 2. The image light passes through the λ / 4 wave plate 21 twice when incident on and exiting the retroreflective plate 2, thereby undergoing polarization conversion from a specific polarization to another polarization. Here, the polarization separation component 101 that selectively reflects image light of a specific polarization has the property of transmitting light of another polarization after polarization conversion, so the image light of a specific polarization after polarization conversion passes through the polarization separation component 101. The image light after passing through the polarization separation component 101 forms a real image of a spatial suspended image 3 on the outside of the transparent component 100. In addition, regarding the main light ray of the image light incident on the retroreflective plate 2, Figure 2A , an example is described in which the incident angle of the principal ray of the image light on the retroreflective plate 2 is 90°. However, the incident angle of the principal ray of the image light on the retroreflective plate 2 is not limited to 90°, and for example, 90°±15° may be used.

[0086] Here, it is explained Figure 2A The first example of polarization design in an optical system. For example, it can be configured so that S-polarized image light is emitted from the display device 1 toward the polarization separation component 101, and the polarization separation component 101 has the characteristic of reflecting S-polarized light and transmitting P-polarized light. In this case, the S-polarized image light reaching the polarization separation component 101 from the display device 1 is reflected by the polarization separation component 101 and travels to the retro-reflective plate 2. When the image light is reflected by the retro-reflective plate 2, it passes twice through the λ / 4 wave plate 21 provided on the incident surface of the retro-reflective plate 2, so that the image light is converted from S-polarized light to P-polarized light. The image light converted to P-polarized light travels again to the polarization separation component 101. Here, because the polarization separation component 101 has the characteristic of reflecting S-polarized light and transmitting P-polarized light, the P-polarized image light passes through the polarization separation component 101 and then through the transparent component 100. The image light that passes through the transparent member 100 is generated by the retroreflective plate 2. Therefore, at a position where the polarization separation member 101 forms a mirror image with respect to the image displayed on the display device 1, an optical image of the image displayed on the display device 1, namely, a spatially suspended image 3, is formed. This polarization design enables the formation of the spatially suspended image 3 to be excellent.

[0087] Next, explain Figure 2AA second example of polarization design in an optical system. For example, a configuration can be made such that P-polarized image light is emitted from the display device 1 toward the polarization separation component 101, and the polarization separation component 101 has the characteristic of reflecting P-polarized light and transmitting S-polarized light. In this case, the P-polarized image light reaching the polarization separation component 101 from the display device 1 is reflected by the polarization separation component 101 and travels to the retro-reflective plate 2. When the image light is reflected by the retro-reflective plate 2, it passes twice through the λ / 4 wave plate 21 provided on the incident surface of the retro-reflective plate 2, so that the image light is converted from P-polarized light to S-polarized light. The image light converted to S-polarized light travels again to the polarization separation component 101. Here, since the polarization separation component 101 has the characteristic of reflecting P-polarized light and transmitting S-polarized light, the S-polarized image light passes through the polarization separation component 101 and then through the transparent component 100. The image light that passes through the transparent member 100 is generated by the retroreflective plate 2. Therefore, at a position where the polarization separation member 101 forms a mirror image with respect to the image displayed on the display device 1, an optical image of the image displayed on the display device 1, namely, a spatially suspended image 3, is formed. This polarization design enables the formation of the spatially suspended image 3 to be excellent.

[0088] Furthermore, the light that forms the spatial suspended image 3 is a collection of light rays that converge from the retroreflective plate 2 to the optical image of the spatial suspended image 3. These light rays continue to travel in a straight line after passing through the optical image of the spatial suspended image 3. Therefore, the spatial suspended image 3 is different from the diffuse image light formed on the screen by a general projector, etc., and is an image with high directivity. Figure 2A In this configuration, when the user views the image 3 from the direction of arrow A, the spatially suspended image 3 is visible as a bright image. However, when other people view the image from the direction of arrow B, the spatially suspended image 3 is not visible at all. This characteristic is particularly suitable for use in systems that display highly secure images, or in systems that display highly confidential images that must be kept secret from people facing the user.

[0089] In addition, depending on the performance of the retroreflector 2, the polarization axis of the reflected image light may not be uniform. Furthermore, the reflection angle may also be non-uniform. Such non-uniform light may not maintain the polarization state and travel angle expected in the design. For example, such light that deviates from the polarization state and travel angle expected in the design may directly re-enter the image display surface side of the liquid crystal display panel 11 from the position of the retroreflector 2 without passing through the polarization separation component. Such light that deviates from the polarization state and travel angle expected in the design may, after reflecting from components within the spatially suspended image display device, re-enter the image display surface side of the liquid crystal display panel 11. Such light that re-enters the image display surface side of the liquid crystal display panel 11 is reflected again on the image display surface of the liquid crystal display panel 11 constituting the display device 1, potentially generating ghost images and reducing the image quality of the spatially suspended image. To this end, in this embodiment, an absorbing polarizer 12 may be provided on the image display surface of the display device 1. Image light emitted from the display device 1 is passed through the absorbing polarizer 12, and reflected light returning from the polarization separation component 101 is absorbed by the absorbing polarizer 12, thereby suppressing the aforementioned re-reflection. This prevents image quality degradation caused by ghosting of the spatially suspended image. Specifically, if the configuration employs an S-polarized image light emitted from the display device 1 to the polarization separation component 101, the absorbing polarizer 12 can be a polarizer that absorbs P-polarized light. Alternatively, if the configuration employs an P-polarized image light emitted from the display device 1 to the polarization separation component 101, the absorbing polarizer 12 can be a polarizer that absorbs S-polarized light.

[0090] The polarization separation member 101 may be formed of, for example, a reflective polarizer or a metal multilayer film that reflects a specific polarization.

[0091] Then, in Figure 2A In (2), an example of the surface shape of a retroreflective sheet is shown as a representative retroreflective sheet 2. It is composed of regularly arranged hexagonal prisms. Light incident on the interior is reflected by the walls and bottom surfaces of the hexagonal prisms to become retroreflected light, which is emitted in a direction corresponding to the incident light and displays a spatial floating image of a real image based on the image displayed on the display device 1.

[0092] The resolution of the spatial floating image depends not only on the resolution of the liquid crystal display panel 11, but also on the resolution of the Figure 2A The outer shape D and pitch P of the retroreflective portion of the retroreflective plate 2 shown in (2) are shown. For example, when using a 7-inch WUXGA (1920×1200 pixels) liquid crystal display panel, even if one pixel (one triplet) is approximately 80 μm, if the diameter D of the retroreflective portion is 240 μm and the pitch is 300 μm, then one pixel of the spatially suspended image is equivalent to 300 μm. Therefore, the effective resolution of the spatially suspended image is reduced to approximately 1 / 3.

[0093] To this end, in order to make the resolution of the spatially floating image the same degree as the resolution of the display device 1, it is preferable to make the diameter and the pitch of the retro-reflective sections close to one pixel of the liquid crystal display panel. On the other hand, in order to suppress moire caused by the retro-reflective plate and the pixels of the liquid crystal display panel, the respective pitches can be designed to deviate by an integer multiple of one pixel. In addition, the shape can be configured such that none of the arbitrary sides of the retro-reflective sections coincide with any of the arbitrary sides of one pixel of the liquid crystal display panel.

[0094] In addition, the surface shape of the retro-reflective plate of the present embodiment is not limited to the above-described example. Various surface shapes that achieve retro-reflection can be adopted. Specifically, a retro-reflective element obtained by periodically arranging a triangular pyramid prism, a hexagonal pyramid prism, another polygonal prism, or a combination thereof can be provided on the surface of the retro-reflective plate of the present embodiment. Alternatively, a retro-reflective element obtained by periodically arranging these prisms and forming a cube corner can be provided on the surface of the retro-reflective plate of the present embodiment. It can also be expressed as an array of corner reflectors, an array of multi-faceted reflectors. Alternatively, a capsule lens type retro-reflective element obtained by periodically arranging glass microbeads can be provided on the surface of the retro-reflective plate of the present embodiment. The detailed structure of these retro-reflective elements can use the related art, and thus a detailed description is omitted. Specifically, the technology disclosed in Japanese Patent Application Publication No. 2001-33609, Japanese Patent Application Publication No. 2001-264525, Japanese Patent Application Publication No. 2005-181555, Japanese Patent Application Publication No. 2008-70898, Japanese Patent Application Publication No. 2009-229942, and the like can be used.

[0095] Other Configuration Example 1 of Optical System of Spatially Floating Image Display Device

[0096] For another configuration example of the optical system of the spatially floating image display device, the configuration of Figure 2B is described. In addition, Figure 2B the structure labeled with the same reference numeral as Figure 2A has the same function and structure as Figure 2A . In order to simplify the description, repeated description is omitted for such a structure.

[0097] Figure 2B The optical system of the spatially floating image display device 1 is the same as that of Figure 2A , and the image light of a specific polarization is output from the display device 1. The image light of the specific polarization output from the display device 1 is input to the polarization separation member 101B. The polarization separation member 101B is a member that selectively transmits the image light of the specific polarization. The polarization separation member 101B is the same as Figure 2AThe polarization separation member 101B is not integrally formed with the transparent member 100 but is independently in a plate shape. Thus, the polarization separation member 101B can also be expressed as a polarization separation plate. The polarization separation member 101B can be configured, for example, as a reflection-type polarizing plate formed by pasting a polarization separation sheet on the transparent member. Alternatively, a metal multilayer film or the like that selectively transmits a specific polarization and reflects a polarization light of another specific polarization can be formed on the transparent member. Figure 2B In this case, the polarization separation member 101B is configured to transmit the image light of the specific polarization output from the display device 1.

[0098] The image light after passing through the polarization separation member 101B is incident on the retroreflective sheet 2. A λ / 4 wave plate 21 is provided on the image light incident surface of the retroreflective sheet. The image light passes through the λ / 4 wave plate 21 twice at the time of incidence and at the time of emission with respect to the retroreflective sheet, and thus is converted in polarization from the specific polarization to another polarization. Here, the polarization separation member 101B has a property of reflecting the polarization light of the another polarization converted by the λ / 4 wave plate 21, and thus the image light after the polarization conversion is reflected on the polarization separation member 101B. The image light after the reflection on the polarization separation member 101B transmits the transparent member 100, and a real image is formed in the space outside the transparent member 100 as the spatially floating image 3.

[0099] Here, a first example of the polarization design in the optical system of Figure 2B For example, the image light of P polarization can be emitted from the display device 1 to the polarization separation member 101B, and the polarization separation member 101B can have a characteristic of reflecting S polarization light and transmitting P polarization light. In this case, the image light of P polarization from the display device 1 reaches the polarization separation member 101B, transmits the polarization separation member 101B, and goes to the retroreflective sheet 2. When the image light is reflected on the retroreflective sheet 2, it passes through the λ / 4 wave plate 21 provided on the incident surface of the retroreflective sheet 2 twice, and thus is converted from P polarization light to S polarization light. The image light after the conversion to S polarization light goes to the polarization separation member 101B again. Here, since the polarization separation member 101B has a characteristic of reflecting S polarization light and transmitting P polarization light, the image light of S polarization is reflected on the polarization separation member 101B and transmits the transparent member 100. The image light after the transmission of the transparent member 100 is the light generated by the retroreflective sheet 2, and thus an optical image of the display image of the display device 1, that is, the spatially floating image 3 is formed at a position where the polarization separation member 101B and the display image of the display device 1 are in a mirror relationship. With such a polarization design, the spatially floating image 3 can be formed well.

[0100] Next, a second example of the polarization design in the optical system of Figure 2Bsecond example of the polarization design in the optical system of FIG. 1. For example, the optical system can be configured such that S-polarized image light is emitted from the display device 1 toward the polarization separation member 101B, and the polarization separation member 101B has a characteristic of reflecting P-polarized light and transmitting S-polarized light. In this case, the S-polarized image light that has reached the polarization separation member 101B from the display device 1 is transmitted through the polarization separation member 101B and goes toward the retroreflective sheet 2. When the image light is reflected on the retroreflective sheet 2, it passes through the λ / 4 wave plate 21 provided on the incident surface of the retroreflective sheet 2 twice, and thus the image light is converted from S-polarized light to P-polarized light. The image light converted to P-polarized light goes toward the polarization separation member 101B again. Here, since the polarization separation member 101B has a characteristic of reflecting P-polarized light and transmitting S-polarized light, the P-polarized image light is reflected on the polarization separation member 101B and is transmitted through the transparent member 100. The image light transmitted through the transparent member 100 is light generated by the retroreflective sheet 2, and thus an optical image of the display image of the display device 1, i.e., the spatially floating image 3, is formed at a position where the polarization separation member 101B and the display image of the display device 1 are in a specular relationship. With such a polarization design, the spatially floating image 3 can be formed well.

[0101] In addition, Figure 2B In the optical system of FIG. 1, the image display surface of the display device 1 and the surface of the retroreflective sheet 2 are arranged in parallel. The polarization separation member 101B is arranged to be inclined by an angle α (e.g., 30°) with respect to the image display surface of the display device 1 and the surface of the retroreflective sheet 2. In this way, when the image light is reflected on the polarization separation member 101B, the direction of travel of the image light reflected on the polarization separation member 101B (the direction of the chief ray of the image light) becomes a direction that differs by an angle β (e.g., 60°) from the direction of travel of the image light incident from the retroreflective sheet 2 (the direction of the chief ray of the image light). By adopting such a structure, the image light is output to the outside of the transparent member 100 at a predetermined angle as illustrated, and the spatially floating image 3 in which an actual image is formed. In the optical system of FIG. 1, Figure 2B In the optical system of FIG. 1, the image display surface of the display device 1 and the surface of the retroreflective sheet 2 are arranged in parallel. The polarization separation member 101B is arranged to be inclined by an angle α (e.g., 30°) with respect to the image display surface of the display device 1 and the surface of the retroreflective sheet 2. In this way, when the image light is reflected on the polarization separation member 101B, the direction of travel of the image light reflected on the polarization separation member 101B (the direction of the chief ray of the image light) becomes a direction that differs by an angle β (e.g., 60°) from the direction of travel of the image light incident from the retroreflective sheet 2 (the direction of the chief ray of the image light). By adopting such a structure, the image light is output to the outside of the transparent member 100 at a predetermined angle as illustrated, and the spatially floating image 3 in which an actual image is formed. In the optical system of FIG. 1, Figure 2B In the structure of FIG. 1, when a user views from the direction of the arrow A, the spatially floating image 3 can be seen as a bright image. However, when another person views from the direction of the arrow B, the spatially floating image 3 cannot be seen at all as an image. This characteristic is very suitable for application in a system that displays an image requiring high security, and a system that displays an image that wants to keep high security from a person directly facing the user.

[0102] As described above, Figure 2B The optical system of FIG. 1 is an optical system of a different structure from the optical system of FIG. 2, but can form a good spatially floating image as with the optical system of FIG. 2. Figure 2A Figure 2A The optical system of FIG. 1 is an optical system of a different structure from the optical system of FIG. 2, but can form a good spatially floating image as with the optical system of FIG. 2.

[0103] ​In addition, an absorption-type polarizing plate can be provided on the side of the transparent member 100 opposite the polarization separation member 101B. This absorption-type polarizing plate can be an absorption-type polarizing plate that transmits the polarized light of the image light from the polarization separation member 101B and absorbs polarized light that is 90° out of phase with the polarization of the image light from the polarization separation member 101B. In this way, the image light used to form the spatially-suspended image 3 is sufficiently transmitted, while reducing the external light incident from the side of the spatially-suspended image 3 of the transparent member 100 by about 50%. As a result, it is possible to reduce the stray light within the optical system of the spatially-suspended image display device that is generated by the external light incident from the side of the spatially-suspended image 3 of the transparent member 100. Figure 2B

[0104] Other Configuration Example 2 of Optical System of Spatially-Suspended Image Display Device

[0105] An other configuration example of the optical system of the spatially-suspended image display device will be described using FIG. 12. In addition, the structures in FIG. 12 that are labeled with the same reference numerals as in FIG. 1 have the same functions and structures as in FIG. 1. Repetitive descriptions will be omitted for such structures. Figure 2C Figure 2C Figure 2B Figure 2B

[0106] Figure 2C The optical system of FIG. 12 differs from the optical system of FIG. 1 only in the configuration angle of the polarization separation member 101B with respect to the image display surface of the display device 1 and the surface of the retroreflective sheet 2. The other structures are the same as in the optical system of FIG. 1, so repetitive descriptions will be omitted. Figure 2B Figure 2B Figure 2C Figure 2B

[0107] In the optical system of FIG. 12, the polarization separation member 101B is configured to be inclined by an angle a with respect to the image display surface of the display device 1 and the surface of the retroreflective sheet 2. Figure 2C Figure 2C ​​​​​​​​​​In this case, the angle a is 45°. With this configuration, the angle β between the traveling direction of the image light incident from the retroreflective sheet 2 and the traveling direction of the image light reflected on the polarization separation member 101B (the direction of the chief ray of the image light) is 90°. With this configuration, the image display surface of the display device 1 and the surface of the retroreflective sheet 2 are in a right angle relationship with the traveling direction of the image light reflected on the polarization separation member 101B, and the angle relationship of the surfaces constituting the optical system can be made simple. If the surface of the transparent member 100 is disposed orthogonal to the traveling direction of the image light reflected on the polarization separation member 101B, the angle relationship of the surfaces constituting the optical system can be made further simple. In Figure 2C In the configuration, when a user views from the direction of the arrow A, the spatially floating image 3 can be seen as a bright image. But when another person views from the direction of the arrow B, the spatially floating image 3 cannot be seen at all as an image. This characteristic is very suitable for application in a system that displays an image requiring high security, and a system that displays an image that wants to keep high security for a person directly facing the user.

[0108] As explained above, Figure 2C the optical system is an optical system of a different configuration from Figure 2A and Figure 2B but can form a good spatially floating image as with Figure 2A and Figure 2B the optical systems. In addition, the angle of the surfaces constituting the optical system can be made simpler.

[0109] In addition, an absorbing polarizing plate can be provided on the side of the polarization separation member 101B of the transparent member 100. This absorbing polarizing plate can be an absorbing polarizing plate that transmits the polarization of the image light from the polarization separation member 101B and absorbs the polarization that is 90° out of phase from the polarization of the image light from the polarization separation member 101B. In this way, the image light used to form the spatially floating image 3 is sufficiently transmitted, while the external light incident from the side of the spatially floating image 3 of the transparent member 100 is reduced by about 50%. Thus, the stray light in the optical system of Figure 2C caused by the external light incident from the side of the spatially floating image 3 of the transparent member 100 can be reduced.

[0110] <Other Configuration Example 3 of the Optical System of the Spatially Floating Image Display Device>

[0111] An other configuration example of the optical system of the spatially floating image display device is explained using Figure 2D Figure 2D the optical system is an optical system of a different configuration from Figures 2A-2C ​The optical system of the different retro-reflective sheet 2 used in the present embodiment. Hereinafter, the optical system of the different retro-reflective sheet 5 used in the present embodiment will be described using the same reference numerals as those of the optical system of the different retro-reflective sheet 2. Figures 2D-2I The other configuration example 3 of the optical system will be described more specifically. Figure 2D The structures denoted by the same reference numerals as those of the configuration example 2 have the same functions and structures as those of the configuration example 2. Repetitive descriptions will be omitted for the sake of simplicity. Figure 2A The structures denoted by the same reference numerals as those of the configuration example 2 have the same functions and structures as those of the configuration example 2. Repetitive descriptions will be omitted for the sake of simplicity. Figures 2A-2C The structures denoted by the same reference numerals as those of the configuration example 2 have the same functions and structures as those of the configuration example 2. Repetitive descriptions will be omitted for the sake of simplicity.

[0112] Figure 2D is a view showing a main part structure of a spatially floating image display device and an example of a retro-reflective part structure of the present embodiment. In the oblique direction of a transparent member 100 such as glass, a display device 10 that emits image light is provided. The display device 10 is provided with a liquid crystal display panel 11 and a light source device 13 that generates light.

[0113] A chief ray 9020 that represents a light beam emitted from the display device 10 travels toward the retro-reflective sheet 5 and is incident on the retro-reflective sheet 5 at an incident angle a. The incident angle a can be, for example, 45° or the like. However, the incident angle a is not limited to 45° and can be, for example, 45° ± 15°.

[0114] The retro-reflective sheet 5 is an optical member having an optical property of retro-reflecting light rays in at least a part of the directions. In addition, the reflected light rays have an optical property of being able to form an image, so the retro-reflective sheet 5 can also be expressed as an imaging optical member or an imaging optical sheet.

[0115] The detailed structure of the retro-reflective sheet 5 will be described using Figure 2E , Figure 2F and the like. Under the action of the retro-reflective sheet 5, the chief ray 9020 travels in the z direction while being retro-reflected in the x and y directions. As a result, the reflected light ray 9021 travels in a direction away from the retro-reflective sheet 5 on an optical path that is mirror-symmetrical with respect to the chief ray 9020 with the retro-reflective sheet 5 as a reference, and passes through the transparent member 100 to form a spatially floating image 3 as a real image on an imaging surface.

[0116] The light beam that forms the spatially-suspended image 3 is a collection of light rays that converge at the optical image of the spatially-suspended image 3 from the retro-reflective sheet 5, and these light rays travel straight after passing through the optical image of the spatially-suspended image 3. Thus, the spatially-suspended image 3 is an image with high directivity, unlike a diffuse image formed on a screen with a general projector or the like. Therefore, in the configuration of Fig. 2, the spatially-suspended image 3 can be seen as a bright image when a user views from the direction of arrow A. However, the spatially-suspended image 3 cannot be seen at all as an image when another person views from the direction of arrow B. This characteristic is suitable for application in a system that displays an image requiring high security, and a system that displays an image with high security that wants to keep secret a person directly facing the user.

[0117] Using Figure 2E 、 Figure 2F An example of the configuration of the retro-reflective sheet 5 will be described. The retro-reflective sheet 5 is a configuration in which a plurality of corner reflectors 9040 are arrayed in an array on the surface of a transparent member 50. It can also be referred to as a corner reflector array or a multi-faceted reflector array. The specific configuration of the corner reflector 9040 will be described using Figure 2G 、 Figure 2H 、 Figure 2I In detail, the light rays 9111, 9112, 9113, 9114 emitted from the light source 9110 are reflected twice by the 2 mirror surfaces 9041, 9042 of the corner reflector 9040, becoming reflected light rays 9121, 9122, 9123, 9124. This 2-time reflection is a retro-reflection in which the direction is folded back (travels in a direction rotated by 180°) in the x, y directions to the same direction as the incident direction, and a regular reflection in which the incident angle and the reflection angle coincide due to total reflection in the z direction.

[0118] That is, the light rays 9111 to 9114 are reflected on a straight line in the z direction with respect to the corner reflector 9040 on which the light rays 9111 to 9114 are symmetrical, becoming the reflected light rays 9121 to 9124, and an aerial real image 9120 is formed. In addition, the light rays 9111 to 9114 emitted from the light source 9110 are 4 light rays that represent diffuse light from the light source 9110, and depending on the diffusion characteristics of the light source 9110, the light rays incident on the retro-reflective sheet 5 are not limited to these, but any incident light rays are reflected similarly, forming the aerial real image 9120. In addition, the position of the light source 9110 and the position of the x direction of the aerial real image 9120 are shown offset for ease of viewing of the drawing, but in fact the position of the light source 9110 and the position of the x direction of the aerial real image 9120 are the same position, and when viewed from the z direction, are coincident positions.

[0119] Next, the configuration of the corner reflector 9040 will be described using Figure 2G 、 Figure 2H 、 Figure 2IThe structure and effects of the corner reflector 9040 that constitutes the retroreflective sheet 5 will be described. The corner reflector 9040 is a rectangular parallelepiped in which only specific two faces are mirror surfaces 9041, 9042, and the other four faces are formed of transparent members. The retroreflective sheet 5 is configured such that the corner reflectors 9040 are arrayed in a manner that the corresponding mirror surfaces face in the same direction.

[0120] When viewed from the top surface (+z direction), the light ray 9111 emitted from the light source 9110 is incident on the mirror surface 9041 (or the mirror surface 9042) at a specific incident angle, and after total reflection occurs at the reflection point 9130, total reflection occurs again at the reflection point 9132 on the mirror surface 9042 (or the mirror surface 9041).

[0121] Let the incident angle of the light ray 9111 on the mirror surface 9041 (or the mirror surface 9042) be θ, then the incident angle of the first reflected light ray 9131 on the mirror surface 9042 (or the mirror surface 9041) after reflection on the mirror surface 9041 (or the mirror surface 9042) can be expressed as 90° - θ. Thus, with respect to the light ray 9111, the second reflected light ray 9121 is rotated by 2θ due to the first reflection, and by 2 x (90° - θ) due to the second reflection, so it becomes a reversed light path that is a total of 180°. On the other hand, when viewed from the side surface (-x and -y directions), total reflection in the z direction occurs only once. Thus, let the incident angle on the mirror surface 9041 or the mirror surface 9042 be φ, then the reflected light 9121 is rotated by 2 x φ with respect to the light ray 9111 due to the first reflection.

[0122] According to the above, the light ray that is incident on the corner reflector 9040 is subjected to retroreflective reflection that becomes a reversed light path in the x and y directions, and is subjected to positive reflection due to total reflection in the z direction. Considering the retroreflective sheet 5, because the same reflection occurs for each light path, a point symmetrical with respect to the z axis is imaged due to the reversed light path that has convergence in the x and y directions.

[0123] Here, Figures 2A-2C In the optical system of the above,

[0124] On the other hand, Figure 2DIn the optical system, the retro-reflective plate 5 has a retro-reflective characteristic in the 2-axial direction, and regular reflection occurs in the other 1-axial direction. Thus, in a case where an incident light beam having diffusivity is incident on the retro-reflective plate 5, a reflected light beam having condensivity, which is reflected by the corner cube array, travels in a direction opposite to a side of the retro-reflective plate 5 where a light source of the incident light beam is located. The reflected light beam having condensivity is aerially imaged to form the spatially floating image 3.

[0125] The direction of travel of the chief ray of the reflected light beam having condensivity, which is reflected by the corner cube array of the retro-reflective plate 5, is not opposite to the direction of travel of the chief ray of the incident light beam having diffusivity, which is incident on the retro-reflective plate 5. The normal direction component of the face of the plate shape of the retro-reflective plate 5 in the direction of travel of the chief ray of the incident light beam having diffusivity, which is incident on the retro-reflective plate 5, and the normal direction component of the face of the plate shape of the retro-reflective plate 5 in the direction of travel of the chief ray after being reflected on the retro-reflective plate 5 to become the reflected light beam having condensivity, do not change before and after the reflection by the corner cube array and travel linearly.

[0126] That is, by the reflection on the retro-reflective plate 5, the incident light beam having diffusivity is converted into the reflected light beam having condensivity, but in the normal direction of the face of the plate shape of the retro-reflective plate 5, the light beam travels through the retro-reflective plate 5. Here, the incident light beam having diffusivity, which is incident on the retro-reflective plate 5, and the reflected light beam having condensivity, which is emitted from the retro-reflective plate 5, have a geometrically symmetrical relationship with respect to the face of the plate shape of the retro-reflective plate 5.

[0127] Regarding the spatially floating image obtained by aerial imaging of the light rays from the image output section 10, the resolution is largely dependent on the resolution of the liquid crystal display panel 11, in addition to the resolution of the retro-reflective plate 5. Figure 2E , Figure 2F The diameter D and the pitch P (not shown) of the retro-reflective portion of the retro-reflective plate 5 shown in FIG. 6. For example, in a case where a 7-inch WUXGA (1920 x 1200 pixels) liquid crystal display panel is used, even if one pixel (one triad) is about 80 μm, if the diameter D of the retro-reflective portion is 240 μm and the pitch P is 300 μm, one pixel of the spatially floating image corresponds to 300 μm. Thus, the effective resolution of the spatially floating image is reduced to about 1 / 3.

[0128] To achieve this, in order to ensure that the resolution of the spatially suspended image is comparable to that of the display device 10, the diameter D and pitch P of the retroreflective portion are preferably close to that of a single pixel of the LCD panel. Furthermore, to suppress moiré patterns caused by the pixels of the retroreflective plate and the LCD panel, the pitch ratio of each can be designed to deviate from an integer multiple of a single pixel. Furthermore, the shape of the retroreflective portion can be configured so that no side of the portion overlaps with any side of a single pixel of the LCD panel.

[0129] In addition, the shape of the retroreflective plate (imaging optical plate) of this embodiment is not limited to the above examples. It can have various shapes that achieve retroreflection. Specifically, it can be various cubic corner reflectors, corner reflector arrays, slit mirror arrays, dihedral corner reflector arrays, multi-faceted reflector arrays, or a combination of these reflective surfaces arranged periodically. Alternatively, a capsule lens-type retroreflective element obtained by periodically disposing glass microbeads can be provided on the surface of the retroreflective plate of this embodiment. The detailed structure of these retroreflective elements can be based on existing technologies, so detailed description is omitted. Specifically, the technologies disclosed in Japanese Patent Application Publication No. 2017-33005, Japanese Patent Application Publication No. 2019-133110, Japanese Patent Application Publication No. 2017-67933, WO2009 / 131128, etc. can be used.

[0130] in addition, Figure 2D In the optical system, the image light emitted from the display device 10 can be in any polarization state. There is no problem whether it is S polarization or P polarization.

[0131] As explained above, Figure 2D The optical system uses Figures 2A-2C The optical system is different from the retro-reflective plate, but with Figures 2A-2C The optical system can also form better spatial suspended images.

[0132] According to the above description Figure 2A 、 Figure 2B 、 Figure 2C 、 Figure 2D The optical system can provide brighter and higher-quality spatial suspended images.

[0133] <<Block diagram of the internal structure of the spatial floating image display device>>

[0134] Next, a block diagram of the internal structure of the spatial floating image display device 1000 will be described. Figure 3 This is a block diagram showing an example of the internal structure of the spatial floating image display device 1000.

[0135] The spatially-suspended image display device 1000 includes a retroreflective portion 1101, an image display portion 1102, a light guide 1104, a light source 1105, a power supply 1106, an external power supply input interface 1111, an operation input portion 1107, a nonvolatile memory 1108, a memory 1109, a control portion 1110, an image signal input portion 1131, a sound signal input portion 1133, a communication portion 1132, an aerial operation detection sensor 1351, an aerial operation detection portion 1350, a sound output portion 1140, a microphone 1139, an image control portion 1160, a storage portion 1170, an imaging portion 1180, and the like. In addition, a removable medium interface 1134, a posture sensor 1113, a transmission-type self-emission image display device 1650, a second display device 1680, a secondary battery 1112, and the like can also be included.

[0136] The various constituent elements of the spatially-suspended image display device 1000 are disposed in a housing 1190. In addition, Figure 3 The imaging portion 1180 and the aerial operation detection sensor 1351 shown can also be disposed outside the housing 1190.

[0137] Figure 3 The retroreflective portion 1101 of the spatially-suspended image display device 1000 corresponds to Figure 2A 、 Figure 2B 、 Figure 2C the retroreflective plate 2. The retroreflective portion 1101 causes light that has been modulated by the image display portion 1102 to undergo retroreflective. Light that is output to the outside of the spatially-suspended image display device 1000 from the reflected light from the retroreflective portion 1101 forms the spatially-suspended image 3.

[0138] Figure 3 The image display portion 1102 of the spatially-suspended image display device 1000 corresponds to Figure 2A 、 Figure 2B 、 Figure 2C the liquid crystal display panel 11. Figure 3 The light source 1105 of the spatially-suspended image display device 1000 corresponds to Figure 2A 、 Figure 2B 、 Figure 2C the light source device 13. Figure 3 The image display portion 1102, the light guide 1104, and the light source 1105 of the spatially-suspended image display device 1000 correspond to Figure 2A 、 Figure 2B 、 Figure 2C the display device 1.

[0139] The image display portion 1102 is a display portion that generates an image by modulating light that has been transmitted based on an image signal that is input under the control of the image control portion 1160, which will be described later. The image display portion 1102 corresponds to Figure 2A 、 Figure 2B 、 Figure 2CThe liquid crystal display panel 11 is a liquid crystal display panel 11 of the IPS (In-Plane Switching) system. As the liquid crystal display panel 11, a liquid crystal display panel 11 of the TN (Twisted Nematic) system or the like can be used. As the liquid crystal display panel 11, a liquid crystal display panel 11 of the transmissive type or the like can be used. As the liquid crystal display panel 11, a liquid crystal display panel 11 of the reflective type or the like can be used.

[0140] The light source 1105 generates light for the image display section 1102, and is a solid light source such as an LED light source or a laser light source. The power supply 1106 converts an AC current input from the outside via the external power supply input interface 1111 into a DC current and supplies the light source 1105 with power. The power supply 1106 also supplies necessary DC currents to each of the parts in the space-suspended image display apparatus 1000. The secondary battery 1112 accumulates power (electric power) supplied from the power supply 1106. In the case where power is not supplied from the outside via the external power supply input interface 1111, the secondary battery 1112 supplies power to the light source 1105 and other structures requiring power. That is, in the case where the space-suspended image display apparatus 1000 is provided with the secondary battery 1112, the space-suspended image display apparatus 1000 can be used by the user even in the case where power is not supplied from the outside.

[0141] The light guide body 1104 guides light generated by the light source 1105 so as to be radiated toward the image display section 1102. The combination of the light guide body 1104 and the light source 1105 can be referred to as a backlight of the image display section 1102. The light guide body 1104 can have a structure mainly using glass. The light guide body 1104 can have a structure mainly using plastic. The light guide body 1104 can have a structure using a mirror. Various ways can be considered as the combination of the light guide body 1104 and the light source 1105. Specific structural examples of the combination of the light guide body 1104 and the light source 1105 will be described later in detail.

[0142] The air operation detection sensor 1351 is a sensor that detects an operation of the finger of the user 230 on the space-suspended image 3. The air operation detection sensor 1351, for example, senses a range overlapping the entire display range of the space-suspended image 3. The air operation detection sensor 1351 can also sense only a range overlapping at least a part of the display range of the space-suspended image 3.

[0143] As a specific example of the air operation detection sensor 1351, a distance sensor configured using non-visible light, non-visible light laser, or ultrasonic waves, or the like can be given. In addition, the air operation detection sensor 1351 can also be configured by combining a plurality of sensors to be able to detect coordinates of a two-dimensional plane. In addition, the air operation detection sensor 1351 can be configured by a LiDAR (Light Detection and Ranging) of a ToF (Time of Flight) system or an image sensor.

[0144] The air operation detection sensor 1351 only needs to be able to perform sensing to detect a touch operation or the like by a finger of the user on an object displayed as the spatial floating image 3. Such sensing can also be performed using existing technology.

[0145] The air operation detection section 1350 acquires a sensing signal from the air operation detection sensor 1351, judges whether or not a finger of the user 230 has come into contact with an object of the spatial floating image 3 based on the sensing signal, and calculates a position (contact position) at which the finger of the user 230 has come into contact with the object, or the like. The air operation detection section 1350 is configured by a circuit such as an FPGA (Field Programmable Gate Array), for example. In addition, a part of the function of the air operation detection section 1350 can also be implemented by software using a spatial operation detection program executed by the control section 1110, for example.

[0146] The air operation detection sensor 1351 and the air operation detection section 1350 can be configured to be built into the spatial floating image display apparatus 1000, but can also be provided externally separately from the spatial floating image display apparatus 1000. In the case of being provided separately from the spatial floating image display apparatus 1000, the air operation detection sensor 1351 and the air operation detection section 1350 are configured to be able to transmit information and signals to the spatial floating image display apparatus 1000 via a wired or wireless communication connection path, an image signal transmission path.

[0147] In addition, the air operation detection sensor 1351 and the air operation detection section 1350 can also be provided separately. By doing so, it is possible to configure a system in which the spatial floating image display apparatus 1000 that does not have an air operation detection function is used as the main body, and in which only the air operation detection function can be optionally added. In addition, it is also possible to separate only the air operation detection sensor 1351, and to build the air operation detection section 1350 into the spatial floating image display apparatus 1000. In the case where it is desired to more freely arrange the air operation detection sensor 1351 or the like with respect to the arrangement position of the spatial floating image display apparatus 1000, a structure in which only the air operation detection sensor 1351 is separated has an advantage.

[0148] The imaging section 1180 is, for example, a camera having an image sensor, and captures an image of a space in the vicinity of the space floating image 3 and / or the face, arm, finger, or the like of the user 230. The imaging section 1180 can be provided in plural. By using a plurality of imaging sections 1180, or by using an imaging section with a depth sensor, the air operation detection section 1350 can be assisted in detecting a touch operation by the user 230 on the space floating image 3. The imaging section 1180 can also be provided separately from the space floating image display apparatus 1000. In the case where the imaging section 1180 is provided separately from the space floating image display apparatus 1000, the imaging section 1180 can be configured to transmit an imaging signal to the space floating image display apparatus 1000 via a wired or wireless communication connection path or the like.

[0149] For example, in the case where the air operation detection sensor 1351 is configured to detect whether an object has intruded into an intrusion detection plane including the display surface of the space floating image 3 as an object intrusion sensor, there is a case where the air operation detection sensor 1351 cannot detect information such as how far an object (for example, a finger of the user) that has not intruded into the intrusion detection plane is from the intrusion detection plane, or how close the object is to the intrusion detection plane.

[0150] In this case, by using information such as depth calculation information of the object based on captured images of a plurality of imaging sections 1180 and depth information of the object obtained by a depth sensor, the distance of the object from the intrusion detection plane can be calculated. Then, this information and various information such as the distance of the object from the intrusion detection plane are used for various display controls with respect to the space floating image 3.

[0151] In addition, the air operation detection sensor 1351 can not be used, and a touch operation by the user 230 on the space floating image 3 can be detected by the air operation detection section 1350 based on captured images of the imaging section 1180.

[0152] In addition, the face of the user 230 operating the space floating image 3 can be captured by the imaging section 1180, and a recognition process of the user 230 can be performed by the control section 1110. In addition, in order to determine whether another person is standing in the periphery or behind the user 230 operating the space floating image 3, or whether the other person is peeping at the operation of the user 230 on the space floating image 3, the imaging section 1180 can also capture an image of a range including the user 230 operating the space floating image 3 and the periphery of the user 230.

[0153] The operation input section 1107 is, for example, a signal receiving section or an infrared receiving section of an operation button or a remote controller, and inputs a signal regarding an operation different from an aerial operation (touch operation) by the user 230. The operation input section 1107 can be used for operation by an administrator in addition to the above-described user 230 who performs a touch operation on the spatial floating image 3.

[0154] The image signal input section 1131 is connected to an external image output device to input image data. The image signal input section 1131 can be various digital image input interfaces. For example, it can be constituted by an image input interface of the HDMI (registered trademark) (High-Definition Multimedia Interface) standard, an image input interface of the DVI (Digital Visual Interface) standard, or an image input interface of the DisplayPort standard, and the like.

[0155] Alternatively, an analog image input interface such as an analog RGB and a component video can be provided. The sound signal input section 1133 is connected to an external sound output device to input sound data. The sound signal input section 1133 can be constituted by a sound input interface of the HDMI standard, an optical digital terminal interface, or a coaxial digital terminal interface, and the like. In the case of adopting an interface of the HDMI standard, the image signal input section 1131 and the sound signal input section 1133 can be constituted as an interface of a terminal and a cable. The sound output section 1140 can output sound based on sound data input to the sound signal input section 1133. The sound output section 1140 can be constituted by a speaker.

[0156] In addition, the sound output section 1140 can also output a built-in operation sound, an error warning sound. Alternatively, a structure that outputs a digital signal to an external device as the sound output section 1140 like an Audio Return Channel function prescribed in the HDMI standard can be adopted. The microphone 1139 is a microphone that collects sound around the spatial floating image display device 1000 and converts it into a signal to generate a sound signal. It can be constituted so that the microphone collects a voice (speech) of a person such as a voice of a user, and a sound recognition process (speech recognition process) is performed on the generated sound signal by the control section 1110 described later, and character information is obtained from the sound signal.

[0157] The non-volatile memory 1108 stores various data used in the spatial floating image display device 1000. Among the data stored in the non-volatile memory 1108, for example, there are data for various operations to be displayed by the spatial floating image 3, data and layout information of display icons, and data of objects for user operation. The memory 1109 stores image data displayed by the spatial floating image 3 and data for control of the device, and the like.

[0158] The control section 1110 controls the operation of each section connected thereto. In addition, the control section 1110 can cooperate with the program stored in the memory 1109, and perform arithmetic processing based on information acquired from each section in the space floating image display device 1000.

[0159] The communication section 1132 communicates with external devices, external servers, and the like via a wired or wireless communication interface. In the case where the communication section 1132 has a wired communication interface, the wired communication interface can be configured using, for example, a LAN interface of the Ethernet standard or the like. In the case where the communication section 1132 has a wireless communication interface, the wireless communication interface can be configured using, for example, a communication interface of the Wi-Fi system, a communication interface of the Bluetooth system, a mobile communication interface of 4G, 5G, or the like. Through communication via the communication section 1132, various data such as image data, sound data, and the like are transmitted and received.

[0160] In addition, the removable medium interface 1134 is an interface that connects a detachable recording medium (removable medium). The detachable recording medium (removable medium) can be configured using a semiconductor element memory such as a solid state drive (SSD), a magnetic recording medium recording device such as a hard disk drive (HDD), or an optical recording medium such as an optical disk. The removable medium interface 1134 can read various information such as various data such as image data, sound data, and the like recorded on the detachable recording medium. The image data, image data, and the like recorded on the detachable recording medium are output as a space floating image 3 via the image display section 1102 and the retroreflective section 1101.

[0161] The storage section 1170 is a storage device that records various information such as various data such as image data, sound data, and the like. The storage section 1170 can be configured using a magnetic recording medium recording device such as a hard disk drive (HDD) or a semiconductor element memory such as a solid state drive (SSD). In the storage section 1170, various information such as various data such as image data, sound data, and the like can be recorded in advance at the time of product shipment, for example. In addition, the storage section 1170 can also record various information such as various data such as image data, sound data, and the like acquired from external devices and external servers and the like via the communication section 1132.

[0162] The image data, image data, and the like recorded in the storage section 1170 are output as a space floating image 3 via the image display section 1102 and the retroreflective section 1101. Image data, image data, and the like of display icons and objects for user operation and the like displayed as a space floating image 3 are also recorded in the storage section 1170.

[0163] Layout information of display icons, objects, and the like displayed as the spatial floating image 3, information of various metadata about the objects, and the like are also recorded in the storage section 1170. Sound data recorded in the storage section 1170 is outputted as sound, for example, from the sound output section 1140.

[0164] The image control section 1160 performs various controls on the image signal inputted to the image display section 1102. The image control section 1160 can also be referred to as an image processing circuit, and can be configured by hardware such as an ASIC, an FPGA, a video processor, and the like. In addition, the image control section 1160 can also be referred to as an image processing section, an image processing unit. The image control section 1160 performs control of image switching, for example, and switches which of the image signal stored in the memory 1109, the image signal (image data) inputted to the image signal input section 1131, and the like is inputted to the image display section 1102.

[0165] In addition, the image control section 1160 can also perform control to generate a superimposed image signal obtained by superimposing the image signal stored in the memory 1109 and the image signal inputted from the image signal input section 1131, and input the superimposed image signal to the image display section 1102, thereby forming a composite image as the spatial floating image 3.

[0166] In addition, the image control section 1160 can also perform control to perform image processing on the image signal inputted from the image signal input section 1131 and the image signal stored in the memory 1109, and the like. As the image processing, there are zoom processing of performing enlargement, reduction, deformation, and the like of the image, brightness adjustment processing of changing the brightness, contrast adjustment processing of changing the contrast curve of the image, Retinex processing of decomposing the image into components of light and changing the weight of each component, and the like.

[0167] In addition, the image control section 1160 can also perform special effect image processing and the like on the image signal inputted to the image display section 1102 for assisting the aerial operation (touch operation) of the user 230. The special effect image processing is performed based on the detection result of the touch operation of the user 230 by the aerial operation detection section 1350 and the captured image of the user 230 by the imaging section 1180, for example.

[0168] The posture sensor 1113 is a sensor composed of a gravity sensor, an acceleration sensor, or a combination thereof, and is capable of detecting the installation posture of the spatially suspended image display device 1000. The control unit 1110 can control the operation of each connected unit based on the posture detection results of the posture sensor 1113. For example, if it is detected that the user's posture is not ideal, the image displayed on the image display unit 1102 can be controlled to stop displaying and an error message can be displayed to the user. Alternatively, if the posture sensor 1113 detects a change in the installation posture of the spatially suspended image display device 1000, the image display unit 1102 can be controlled to rotate its display direction.

[0169] As described above, the spatial floating image display device 1000 is equipped with various functions. However, the spatial floating image display device 1000 does not need to have all of these functions and can be any structure as long as it has the function of forming the spatial floating image 3.

[0170] <Structure example of a spatial floating image display device>

[0171] Next, the configuration example of the spatial floating image display device is described. Regarding the layout of the components of the spatial floating image display device of this embodiment, there can be various layouts according to the usage form. Figures 4A-4M The respective layouts are described. Figures 4A-4M In any of the examples, the bold line surrounding the spatial floating image display device 1000 represents an example of the housing structure of the spatial floating image display device 1000 .

[0172] Figure 4A This is a diagram showing an example of the structure of a spatial floating image display device. Figure 4A The space suspended image display device 1000 shown is equipped with Figure 2A The optical system corresponds to the optical system. Figure 4A The spatial floating image display device 1000 is arranged horizontally with the surface forming the spatial floating image 3 facing upward. Figure 4A In the figure, the transparent component 100 of the spatial suspended image display device 1000 is set on the top surface of the device. Compared with the surface of the transparent component 100 of the spatial suspended image display device 1000, the spatial suspended image 3 is formed above. The light of the spatial suspended image 3 travels in an obliquely upward direction. When the air operation detection sensor 1351 is set as shown in the figure, the operation of the user 230's finger on the spatial suspended image 3 can be detected. In addition, the x direction is the left and right direction from the user's perspective, the y direction is the front and back direction (depth direction) from the user's perspective, and the z direction is the up and down direction (vertical direction). Below, Figures 4A-4MThe definitions of the x direction, the y direction, and the z direction in each of the drawings are the same, so repeated explanations are omitted.

[0173] Figure 4B is a diagram showing an example of the structure of a spatially-suspended image display device. Figure 4B The spatially-suspended image display device 1000 shown in Figure 2A corresponding to the optical system of Figure 4B The spatially-suspended image display device 1000 shown in is disposed vertically with the side on which the spatially-suspended image 3 is formed facing upward. That is, Figure 4B In the spatially-suspended image display device 1000, the transparent member 100 is disposed on the front surface of the device (in the direction of the user 230). The spatially-suspended image 3 is formed on the side of the user 230 compared with the surface of the transparent member 100 of the spatially-suspended image display device 1000. The light of the spatially-suspended image 3 travels in the obliquely upward direction. In the case where the in-air operation detection sensor 1351 is disposed as shown in the drawing, the operation of the user's finger on the spatially-suspended image 3 can be detected. Here, as shown in Figure 4B The in-air operation detection sensor 1351 can use the reflection of the sensing light by the user's fingernail for touch detection by sensing the user's finger from the upper side. Generally, the reflectance of the fingernail is higher than that of the finger pad, so the accuracy of touch detection can be improved by such a structure.

[0174] Figure 4C is a diagram showing an example of the structure of a spatially-suspended image display device. Figure 4C The spatially-suspended image display device 1000 shown in Figure 2B corresponding to the optical system of Figure 4C The spatially-suspended image display device 1000 shown in is disposed horizontally with the side on which the spatially-suspended image 3 is formed facing upward. That is, Figure 4C In the spatially-suspended image display device 1000, the transparent member 100 is disposed on the top surface of the device. The spatially-suspended image 3 is formed on the upward direction compared with the surface of the transparent member 100 of the spatially-suspended image display device 1000. The light of the spatially-suspended image 3 travels in the obliquely upward direction. In the case where the in-air operation detection sensor 1351 is disposed as shown in the drawing, the operation of the user's finger on the spatially-suspended image 3 can be detected.

[0175] Figure 4D is a diagram showing an example of the structure of a spatially-suspended image display device. Figure 4D The spatially-suspended image display device 1000 shown in Figure 2B corresponding to the optical system of Figure 4DThe spatially-suspended image display device 1000 shown is disposed vertically with the side on which the spatially-suspended image 3 is formed facing the front of the spatially-suspended image display device 1000 (in the direction of the user 230). That is, Figure 4D In this case, the transparent member 100 of the spatially-suspended image display device 1000 is disposed on the front of the device (in the direction of the user 230). The spatially-suspended image 3 is formed on the side of the user 230 compared to the face of the transparent member 100 of the spatially-suspended image display device 1000. The light of the spatially-suspended image 3 travels in the obliquely upward direction. In the case where the air operation detection sensor 1351 is disposed as shown, the operation of the user's finger on the spatially-suspended image 3 can be detected. Here, as shown, the air operation detection sensor 1351 can use the reflection of the sensing light by the user's fingernail for touch detection by sensing the user's finger from the upper side. Generally, the reflectance of the fingernail is higher than that of the finger pad, so the precision of touch detection can be improved by such a structure. Figure 4D

[0176] Figure 4E is a diagram showing an example of the structure of a spatially-suspended image display device. Figure 4E The spatially-suspended image display device 1000 shown is equipped with an optical system corresponding to the optical system of Figure 2C Figure 4E The spatially-suspended image display device 1000 shown is disposed horizontally with the side on which the spatially-suspended image 3 is formed facing upward. That is, Figure 4E In this case, the transparent member 100 of the spatially-suspended image display device 1000 is disposed on the top surface of the device. The spatially-suspended image 3 is formed upward compared to the face of the transparent member 100 of the spatially-suspended image display device 1000. The light of the spatially-suspended image 3 travels in the directly upward direction. In the case where the air operation detection sensor 1351 is disposed as shown, the operation of the user's finger on the spatially-suspended image 3 can be detected.

[0177] Figure 4F is a diagram showing an example of the structure of a spatially-suspended image display device. Figure 4F The spatially-suspended image display device 1000 shown is equipped with an optical system corresponding to the optical system of Figure 2C Figure 4F The spatially-suspended image display device 1000 shown is disposed vertically with the side on which the spatially-suspended image 3 is formed facing the front of the spatially-suspended image display device 1000 (in the direction of the user 230). That is, Figure 4F ​​​In the spatial floating image display device 1000, the transparent member 100 is positioned on the front of the device (facing the user 230). The spatial floating image 3 is formed on the user 230 side relative to the surface of the transparent member 100 of the spatial floating image display device 1000. Light from the spatial floating image 3 travels in the direction in front of the user. When the mid-air operation detection sensor 1351 is positioned as shown, it is possible to detect finger manipulation of the spatial floating image 3 by the user 230.

[0178] Figure 4G This is a diagram showing an example of the structure of a spatial floating image display device. Figure 4G The space suspended image display device 1000 shown is equipped with Figure 2C The optical system corresponds to the optical system. Figures 4A-4F In the optical system of the spatial floating image display device, the optical path of the center of the image light emitted from the display device 1 is located on the yz plane. Figures 4A-4F In the optical system of the spatial floating image display device, the image light travels in the front-back direction and the up-down direction from the user's perspective. Figure 4G In the optical system of the spatial floating image display device shown in FIG, the optical path of the center of the image light emitted from the display device 1 is located on the xy plane. Figure 4G In the optical system of the spatial floating image display device shown, image light travels in the left-right direction and the front-back direction as viewed by the user. Figure 4G The spatial floating image display device 1000 is arranged such that the surface forming the spatial floating image 3 faces the front of the device (the direction of the user 230). Figure 4G In the spatial floating image display device 1000, the transparent member 100 is positioned on the front of the device (facing the user 230). The spatial floating image 3 is formed on the user's side relative to the surface of the transparent member 100 of the spatial floating image display device 1000. Light from the spatial floating image 3 travels in the direction in front of the user. When the mid-air operation detection sensor 1351 is positioned as shown, it is possible to detect finger manipulation of the spatial floating image 3 by the user 230.

[0179] Figure 4H This is a diagram showing an example of the structure of a spatial floating image display device. Figure 4H The spatial floating image display device 1000 has a window made of a transparent plate 100B such as glass or plastic on the back side of the device (the side opposite to the position where the user 230 views the spatial floating image 3, that is, the side opposite to the direction of travel of the image light of the spatial floating image 3 toward the user 230). Figure 4G The other structures are different from the space suspended image display device. Figure 4G The structure of the spatial suspended image display device is the same as that of the spatial suspended image display device, so repeated description is omitted.Figure 4H In the spatially-suspended image display device 1000 of

[0180] In addition, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, there is a possibility that a part of the image light output from the display device 1 is reflected on the polarization separation member 101B and goes to the transparent plate 100B. And depending on the coating performance of the surface of the transparent plate 100B, there is a possibility that the light is reflected again on the surface of the transparent plate 100B to become stray light seen by the user. Therefore, in order to prevent this stray light, the transparent plate 100B can not be provided at the above-mentioned window of the device back of the spatially-suspended image display device 1000.

[0181] Figure 4I is a diagram showing an example of the structure of a spatially-suspended image display device. Figure 4I The spatially-suspended image display device 1000 of Figure 4H The spatially-suspended image display device of Figure 4H The spatially-suspended image display device 1000 of

[0182] Figure 4I The opening and closing door 1410 of the spatially-suspended image display device 1000 has, for example, a light shielding plate, includes a mechanism that moves (slides) the light shielding plate, a mechanism that rotates, or a mechanism that makes the light shielding plate detachable, whereby the window (back side window) of the transparent plate 100B located on the deep side of the spatially-suspended image display device 1000 can switch between the open state and the light shielding state. The opening and closing door 1410 can be driven by a motor not shown to move (slide) and rotate the light shielding plate in an electric manner. The motor can be controlled by the control section 1110 of Figure 3 Figure 4I The example of

[0183] ​For example, in a case where the scenery visible in the depth of the window of the transparent plate 100B of the spatially-suspended image display apparatus 1000 is the outdoor, the brightness of the sunlight varies depending on the weather. When the sunlight outside is strong, there is a case where the background of the spatially-suspended image 3 becomes too bright and the recognizability of the spatially-suspended image 3 by the user 230 decreases. In such a case, if the light-shielding plate of the opening and closing door 1410 is moved (slid), rotated, or installed so as to make the back side window a light-shielded state, the background of the spatially-suspended image 3 becomes dark, so the recognizability of the spatially-suspended image 3 can be relatively improved. The light-shielding operation of the light-shielding plate of the opening and closing door 1410 can also be performed manually by the user 230 directly. The light-shielding operation of the light-shielding plate of the opening and closing door 1410 can also be performed in accordance with an operation input by the operation input section 1107, and the control section 1110 can control a not-illustrated motor to perform the light-shielding operation of the light-shielding plate of the opening and closing door 1410 in accordance with the operation input by the operation input section 1107. Figure 3

[0184] In addition, a light intensity sensor can also be provided on the back side (the opposite side of the user 230) of the spatially-suspended image display apparatus 1000, for example, in the vicinity of the back side window, and the brightness of the space outside the back side window can be measured. In this case, the opening and closing operation of the light-shielding plate of the opening and closing door 1410 can be controlled by the control section 1110 of the control section 1110 in accordance with the detection result of the light intensity sensor. By thus controlling the opening and closing operation of the light-shielding plate of the opening and closing door 1410, even if the user 230 does not manually perform the opening and closing operation of the light-shielding plate of the opening and closing door 1410, the recognizability of the spatially-suspended image 3 can be more favorably maintained. Figure 3

[0185] In addition, the light-shielding plate of the opening and closing door 1410 can also be detachably attached. The user can select whether to make the back side window an open state or a light-shielded state in accordance with the use purpose of the spatially-suspended image display apparatus 1000 and the setting environment. If it is intended to use the back side window in a light-shielded state for a long time, the detachable light-shielding plate can be fixed in a light-shielded state. In addition, if it is intended to use the back side window in an open state for a long time, the use can be made while the detachable light-shielding plate is detached. The attachment and detachment of the light-shielding plate can be performed using a screw, a hook structure, or an embedding structure.

[0186] In addition, the light-shielding plate of the opening and closing door 1410 can also be detachably attached. The user can select whether to make the back side window an open state or a light-shielded state in accordance with the use purpose of the spatially-suspended image display apparatus 1000 and the setting environment. If it is intended to use the back side window in a light-shielded state for a long time, the detachable light-shielding plate can be fixed in a light-shielded state. In addition, if it is intended to use the back side window in an open state for a long time, the use can be made while the detachable light-shielding plate is detached. The attachment and detachment of the light-shielding plate can be performed using a screw, a hook structure, or an embedding structure. Figure 4I ​​The same applies to the example of the spatially-suspended image display device 1000, depending on the polarization distribution of the image light output from the display device 1 and the performance of the polarization separation member 101B, there is a possibility that a portion of the image light output from the display device 1 is reflected on the polarization separation member 101B and goes to the transparent plate 100B. Depending on the coating performance of the surface of the transparent plate 100B, there is a possibility that the light is reflected again on the surface of the transparent plate 100B to become stray light that is seen by the user. Therefore, in order to prevent this stray light, the above-mentioned window on the back of the spatially-suspended image display device 1000 can not be provided with the transparent plate 100B. The above-mentioned opening and closing door 1410 can be provided on the window without the transparent plate 100B. In order to prevent this stray light, the surface on the inside of the housing of the light shielding plate of the above-mentioned opening and closing door 1410 is preferably provided with a coating or a material having a low light reflectance.

[0187] Figure 4J is a diagram showing an example of the structure of a spatially-suspended image display device. Figure 4J The spatially-suspended image display device 1000 of Figure 4H The spatially-suspended image display device differs from the spatially-suspended image display device of Figure 4H The spatially-suspended image display device of

[0188] That is, the liquid crystal shutter can control the transmission of light by voltage control of the liquid crystal element sandwiched between two polarizing plates. Thus, if the liquid crystal shutter is controlled to increase the transmittance, the background of the spatially-suspended image 3 becomes a state in which the scenery through the back side window can be seen. In addition, if the liquid crystal shutter is controlled to increase the transmittance, the background of the spatially-suspended image 3 can become a state in which the scenery through the back side window cannot be seen.

[0189] In addition, the liquid crystal shutter can perform control of intermediate gradations, so a state in which the transmittance is 50%, for example, can also be adopted. For example, the transmittance of the electrically-controlled transmittance variable device 1620 can be controlled by the control section 1110 in accordance with an operation input by the operation input section 1107. Figure 3

[0190] ​In addition, an illuminance sensor can be provided on the back side (the side opposite the user 230) of the spatially floating image display device 1000, for example, near the back side window, to measure the brightness of the space outside the back side window. In this case, the transmittance of the electrically controlled transmittance variable device 1620 can be controlled by the control section 1110 based on the detection result of the illuminance sensor, so the legibility of the spatially floating image 3 can be maintained even better, even if the user 230 does not make an operation input via the operation input section 1107. Figure 3 Figure 3 In addition, the user 230 can make an operation input via the operation input section 1107 to control the transmittance of the electrically controlled transmittance variable device 1620, so the legibility of the spatially floating image 3 can be maintained even better.

[0191] In addition, in the above example, a liquid crystal shutter is described as an example of the electrically controlled transmittance variable device 1620. However, an electronic paper can also be used as another example of the electrically controlled transmittance variable device 1620. The same effects as described above can be obtained even if an electronic paper is used. In addition, the power consumption for maintaining the intermediate gradation state is very small with an electronic paper. Therefore, a spatially floating image display device with low power consumption can be realized compared to the case in which a liquid crystal shutter is used.

[0192] Figure 4K is a diagram showing an example of the structure of a spatially floating image display device. Figure 4K The spatially floating image display device 1000 of Figure 4G differs from the spatially floating image display device of Figure 4G in that a transmissive self-luminous image display device 1650 is provided instead of the transparent member 100. The other structure is the same as that of the spatially floating image display device of

[0193] In the spatially floating image display device 1000 of Figure 4K , the image light beam forms the spatially floating image 3 outside the spatially floating image display device 1000 after passing through the display surface of the transmissive self-luminous image display device 1650. That is, when an image is displayed using the transmissive self-luminous image display device 1650 as a two-dimensional flat display, the spatially floating image 3 can be displayed as a floating image in front of the user from the image of the transmissive self-luminous image display device 1650. At this time, the user 230 can see two images with different depths at the same time. The transmissive self-luminous image display device 1650 can be configured using, for example, a transmissive organic EL panel or the like disclosed in Japanese Patent Application Publication No. 2014-216761 or the like. In addition, the transmissive self-luminous image display device 1650 is not illustrated in Figure 3 , but can be connected to the control section 1110 and other processing sections as one component of the spatially floating image display device 1000 of Figure 3 . ​

[0194] Here, it is possible to achieve an effect that both the background and the character and the like are displayed on the transmissive self-emitting image display device 1650, and then only the character and the like is moved to the space floating image 3 on the user's front side, and thus it is possible to provide the user 230 with a more effective "surprise effect" image experience.

[0195] In addition, if the inside of the space floating image display device 1000 is in a light-shielded state, the background of the transmissive self-emitting image display device 1650 is dark enough. Therefore, in a case where the display device 1 does not display an image or the light source of the display device 1 does not emit light, and only the transmissive self-emitting image display device 1650 displays an image, the transmissive self-emitting image display device 1650 appears to the user 230 not as a transmissive display but as a general two-dimensional flat display (the space floating image 3 in the embodiment of the present application is displayed as a real image in a space where a screen does not exist, so if the light source of the display device 1 is made not to emit light, the predetermined display position of the space floating image 3 is a space where nothing exists). Therefore, by suddenly displaying the character and the object and the like in the air as the space floating image 3 when the transmissive self-emitting image display device 1650 is used to display an image as if it were a general two-dimensional flat display, it is possible to provide the user 230 with a more effective "surprise effect" image experience.

[0196] In addition, the darker the inside of the space floating image display device 1000, the more the transmissive self-emitting image display device 1650 appears as a two-dimensional flat display. Therefore, it is possible to provide an absorbing polarizing plate (not shown) that transmits the polarization of the image light reflected on the polarization separation member 101B and absorbs a polarization that is 90° out of phase with the polarization on the surface (the incident surface on which the image light reflected on the polarization separation member 101B is incident on the transmissive self-emitting image display device 1650, that is, the surface of the transmissive self-emitting image display device 1650 on the side opposite the space floating image 3) on the inside of the space floating image display device 1000 side of the transmissive self-emitting image display device 1650. In this case, the influence on the image light that forms the space floating image 3 is not large, but it is possible to greatly reduce the light that is incident from the outside on the inside of the space floating image display device 1000 via the transmissive self-emitting image display device 1650, and it is possible to make the inside of the space floating image display device 1000 darker, and thus it is more preferable.

[0197] Figure 4L FIG. 1 is a diagram showing an example of the structure of a space floating image display device. Figure 4L The space floating image display device 1000 of FIG. 1 is Figure 4Ka modification of the spatially-suspended image display device. The configuration direction of the structure in the spatially-suspended image display device 1000 is different from that of the spatially-suspended image display device 1000 of Figure 4K is similar to that of the spatially-suspended image display device 1000 of Figure 4F . As to the function, operation, and the like of each structure, the same structure as that of the spatially-suspended image display device 1000 of Figure 4K is omitted.

[0198] Figure 4L The spatially-suspended image display device 1650 is also the same, and the light beam of the image light forms the spatially-suspended image 3 at a position deeper than the transmissive self-emitting image display device 1650 on the side of the user 230.

[0199] Whether it is the example of the spatially-suspended image display device 1000 of Figure 4K or the example of the spatially-suspended image display device 1000 of Figure 4L , the spatially-suspended image 3 is displayed overlapping in front of the image of the transmissive self-emitting image display device 1650 (on the side in front of the user) from the viewpoint of the user 230. Here, the position of the spatially-suspended image 3 and the position of the image of the transmissive self-emitting image display device 1650 differ in the depth direction. Therefore, when the user moves the head (the position of the viewpoint), the user can recognize the depth of the two images due to the parallax. Therefore, by displaying two images having different depth positions, the user can be provided with a better image experience of naked-eye 3D without the need for stereoscopic glasses or the like.

[0200] Figure 4M is a diagram showing an example of the structure of the spatially-suspended image display device. In the spatially-suspended image display device 1000 of Figure 4M , a second display device 1680 is provided on the deep side from the viewpoint of the user, with respect to the polarization separation member 101B of the spatially-suspended image display device 1000 of Figure 4G . The other structures are the same structures as those of the spatially-suspended image display device 1000 of Figure 4G , and the repeated description is omitted.

[0201] Figure 4M In the example of the structure shown in FIG. 17, the second display device 1680 is provided on the deep side of the display position of the spatially-suspended image 3, and the image display surface faces the spatially-suspended image 3. With this structure, from the viewpoint of the user 230, the image of the second display device 1680 and the spatially-suspended image 3, which are two images displayed at different depth positions, can be seen overlapping. That is, it can be considered that the second display device 1680 is configured in a direction in which the image is displayed on the side facing the user 230 who views the spatially-suspended image 3. In addition, the second display device 1680 is not illustrated in FIG. 17, but can be included in the spatially-suspended image display device 1000 of Figure 3 Figure 3 ​is one of the constituent parts of the spatially-suspended image display apparatus 1000, and is connected to other processing sections such as the control section 1110.

[0202] In addition, Figure 4M The image light of the second display apparatus 1680 of the spatially-suspended image display apparatus 1000 is seen by the user 230 after passing through the polarization separation member 101B. Thus, in order to make the image light of the second display apparatus 1680 pass through the polarization separation member 101B better, it is preferable that the image light output from the second display apparatus 1680 be polarized light of a vibration direction that is more suitable for passing through the polarization separation member 101B. That is, it is preferable that the polarization of the image light output from the second display apparatus 1680 be the same as the polarization of the image light output from the display apparatus 1. For example, in the case where the image light output from the display apparatus 1 is S-polarized light, it is preferable that the image light output from the second display apparatus 1680 also be S-polarized light. In addition, in the case where the image light output from the display apparatus 1 is P-polarized light, it is preferable that the image light output from the second display apparatus 1680 also be P-polarized light.

[0203] Figure 4M The example of the spatially-suspended image display apparatus also displays a second image on the deep side of the spatially-suspended image 3, and in this respect has the same effect as Figure 4K The example of the spatially-suspended image display apparatus and Figure 4L The example of the spatially-suspended image display apparatus has the same effect as the example of the spatially-suspended image display apparatus Figure 4K The example of the spatially-suspended image display apparatus and Figure 4L The example of the spatially-suspended image display apparatus differs from the example of the spatially-suspended image display apparatus Figure 4M In the example of the spatially-suspended image display apparatus, the light beams of the image light used to form the spatially-suspended image 3 do not pass through the second display apparatus 1680. Thus, the second display apparatus 1680 does not need to be a transmissive self-luminous image display apparatus, and can be a liquid crystal display that is a two-dimensional flat panel display. The second display apparatus 1680 can also be an organic EL display. Thus, the example of the spatially-suspended image display apparatus differs from the example of the spatially-suspended image display apparatus Figure 4K The example of the spatially-suspended image display apparatus and Figure 4L The example of the spatially-suspended image display apparatus can realize the spatially-suspended image display apparatus 1000 at a lower cost than the example of the spatially-suspended image display apparatus Figure 4M

[0204] Here, depending on the polarization distribution of the image light output from the display apparatus 1 and the performance of the polarization separation member 101B, there is a possibility that a portion of the image light output from the display apparatus 1 is reflected on the polarization separation member 101B and goes to the second display apparatus 1680. There is a possibility that this light (a portion of the image light) is again reflected on the surface of the second display apparatus 1680 to become stray light that is seen by the user.

[0205] ​Therefore, in order to prevent the stray light, an absorption-type polarizing plate can be provided on the surface of the second display device 1680. In this case, the absorption-type polarizing plate can be one that transmits the polarized light of the image light output from the second display device 1680 and absorbs the polarized light that is 90° out of phase with the polarized light of the image light output from the second display device 1680. In addition, in the case where the second display device 1680 is a liquid crystal display, there is an absorption-type polarizing plate on the image exit side inside the liquid crystal display. However, in the case where the exit surface of the absorption-type polarizing plate on the image exit side inside the liquid crystal display has a glass cover (a glass cover on the image display surface side), it will not be possible to prevent the stray light caused by the reflection of the light from the outside of the liquid crystal display on the glass cover. Therefore, it is necessary to additionally provide the above-described absorption-type polarizing plate on the surface of the glass cover.

[0206] In addition, when displaying an image using the second display device 1680 that is a two-dimensional flat display, it is possible to display the spatially floating image 3 as an image on the side (the user-front side) of the image of the second display device 1680 that is more in front of the user. At this time, the user 230 can see two images that differ in depth position at the same time. By displaying a character with the spatially floating image 3 and displaying a background on the second display device 1680, it is possible to provide the user 230 with an effect as if the user 230 can see the space in which the character is located in three dimensions.

[0207] In addition, it is possible to achieve an effect in which both an object such as a background and a character are displayed on the second display device 1680, and then only the character or the like is moved to the spatially floating image 3 on the user-front side, whereby it is possible to provide the user 230 with an image experience of a more effective "surprise effect".

[0208] Next, Figure 4N is a diagram that shows an example of the structure of a spatially floating image display device. Figure 4N The spatially floating image display device 1000 of Figure 2D is a spatially floating image display device that employs the optical system of Figures 2A-2C As in the example of the spatially floating image display device that employs the optical system of

[0209] As in the example of the spatially floating image display device that employs the optical system of Figures 2A-2C Figure 2D ​In the example of the spatial floating image display device of the optical system, the spatial floating image 3 is formed in front of the transparent member 100 (on the side close to the user), and the user's finger operation on the spatial floating image 3 can be detected using the sensing light of the air operation detection sensor 1351 arranged on the deep side of the transparent member 100 as seen from the user. Figure 2D In the spatial suspended image display device of the optical system, a transparent member 100 is configured on the deep side thereof as viewed from the user. Figures 2A-2C Compared with the optical system of the spatial suspended image display device, the optical system is different.

[0210] However, in terms of ease of use from the user's perspective, the Figures 2A-2C The spatial suspended image display device of the optical system has the characteristics of Figure 4O The optical system of the spatial suspended image display device is roughly the same in ease of use.

[0211] then, Figure 4O This is a diagram showing an example of the structure of a spatial floating image display device. Figure 4N is Figure 4O FIG. 1 is a diagram visually showing the structure of the internal optical system of the spatial floating image display device 1000. Figure 2D The space suspended image display device 1000 shown is equipped with Figure 4O The optical system corresponds to the optical system. Figure 4O The spatial floating image display device 1000 is shown in a horizontal position with the surface thereof forming the spatial floating image 3 facing upward.

[0212] Right now, Figure 4O In the figure, the transparent member 100 of the spatial floating image display device 1000 is positioned on the top surface of the device. The spatial floating image 3 is formed above the surface of the transparent member 100 of the spatial floating image display device 1000. Light from the spatial floating image 3 travels diagonally upward. When the mid-air operation detection sensor 1351 is positioned as shown, it is possible to detect finger manipulation of the spatial floating image 3 by the user 230.

[0213] Here, for Figure 4A The structure and Figure 4A Compare the structures to identify the differences. Figure 4O In the embodiment, the display device 1 and the spatial floating image 3 have a plane-symmetrical relationship with respect to the surface of the polarization separation component 101. Figure 4A In the embodiment, the display device 1 and the spatial floating image 3 have a plane-symmetrical relationship with respect to the surface of the retroreflective plate 5. Figure 4O The structure of the present invention includes a retro-reflective plate 2 and a λ / 4 wave plate 21, but Figure 4A They do not exist in . In addition,Figure 4O It is more preferable to have an absorbing polarizer 12, but Figure 4A The absorbing polarizer 12 is not particularly required.

[0214] That is, in order to Figure 2A In the structure Figure 2D The optical system is replaced by Figure 4O The optical system is replaced by Figure 4A The structure can be operated as follows. Figure 4A The polarization separation component 101 in the structure is replaced by the retroreflective plate 5, Figures 4A-4G The structure can be modified by removing the retro-reflective plate 2 and the λ / 4 wave plate 21. The absorption polarizer 12 is optional. Figures 2A-2C The structure of the space suspended image display device is equipped with Figure 2D The optical system is replaced by Figure 2D The optical system is replaced by one equipped with Figure 4A The optical system of the space suspended image display device. Figures 4B-4G and Figures 4A-4G The polarization separation component 101 can be replaced by the retroreflective plate 5. Figure 2D The polarization separation component 101B can be replaced by the retroreflective plate 5.

[0215] In this way, it can be achieved Figure 2D The optical system is replaced by Figures 4A-4G The spatial suspended image display device is obtained by replacing the optical system of Figure 5 In the space floating image display device of the optical system, it can also be realized with Figure 5 The spatial suspended image display device has roughly the same ease of use as the spatial suspended image display device.

[0216] <Display device>

[0217] Next, the display device 1 of this embodiment will be described with reference to the drawings. The display device 1 of this embodiment includes an image display element 11 (liquid crystal display panel) and a light source device 13 constituting its light source. Figure 1 , the light source device 13 is shown in an exploded perspective view together with the liquid crystal display panel.

[0218] The liquid crystal display panel (image display element 11) is as follows Figure 5As shown by the arrow 30 in the middle, the light source device 13, which serves as the backlight device, receives an illumination beam having narrow-angle diffusion characteristics, that is, strong directivity (straight travel) and a uniform polarization plane in one direction, similar to laser characteristics. The liquid crystal display panel (image display element 11) modulates the received illumination beam according to the input image signal. The modulated image light is reflected by the retroreflective plate 2 and passes through the transparent member 100 to form a real image in space (refer to FIG. Figure 5 ).

[0219] in addition, Figure 6 The display device 1 includes a liquid crystal display panel 11, a light direction conversion panel 54 for controlling the directional characteristics of the light beam emitted from the light source device 13, and a narrow angle diffuser (not shown) as needed. That is, polarizing plates are provided on both sides of the liquid crystal display panel 11, and the intensity of the light is modulated according to the image signal to emit image light of a specific polarization (see Figure 7 As a result, the desired image is projected onto the retroreflective plate 2 as light of a specific polarization with high directivity (straight travel) via the light direction changing panel 54. After being reflected on the retroreflective plate 2, the light is transmitted through the retroreflective plate 2 to the eyes of the viewer outside the store (space), forming a spatial floating image 3. In addition, a protective cover 50 (see FIG. 5 ) may be provided on the surface of the light direction changing panel 54. Figure 6 、 Figure 6 ).

[0220] <Example 1 of Display Device>

[0221] ​ An example of a specific structure of the display device 1 is shown. ​ exist FIG. 5 The light source device 13 is provided with a liquid crystal display panel 11 and a light direction changing panel 54. FIG. 5 The box shown is formed of plastic, etc., and contains LED elements 201 and light guide 203. FIG. 5 As shown in the figure, a lens shape is provided to convert the divergent light from each LED element 201 into a roughly parallel beam. The cross-sectional area of ​​the lens gradually increases as it goes toward the surface opposite the light receiving portion, and has the effect of causing multiple total reflections of the light as it propagates inside, thereby gradually reducing the divergence angle. The liquid crystal display panel 11 constituting the display device 1 is mounted on the upper surface of the display device 1. In addition, an LED substrate 202 is mounted on one side surface of the housing of the light source device 13 (in this case, the left end surface), on which the LED (Light Emitting Diode) element 201 as a semiconductor light source and its control circuit are mounted. A heat sink, a component for cooling the heat generated by the LED element and the control circuit, can be mounted on the outer surface of the LED substrate 202.

[0222] In addition, on a frame (not shown) of the liquid crystal display panel installed on the upper surface of the housing of the light source device 13, the liquid crystal display panel 11 installed on the frame, and an FPC (Flexible Printed Circuits) (not shown) electrically connected to the liquid crystal display panel 11, and the like are installed. That is, the liquid crystal display panel 11 as an image display element and the LED element 201 as a solid light source generate a display image by modulating the intensity of the transmitted light based on a control signal from an image control section 1160 of a control circuit (not shown) constituting an electronic device. FIG. 3 At this time, the generated image light has a narrow diffusion angle and only a specific polarization component, so it is close to a surface-emitting laser image source driven by an image signal, and a novel image display device that has not been available in the past can be obtained. In addition, under the current circumstances, it is technically and safely impossible to obtain a laser beam of the same size as the image obtained by the above-described display device 1 by a laser device. Therefore, in the present embodiment, for example, a light beam emitted by a general light source having an LED element is used to obtain the above-described light close to a surface-emitting laser image light.

[0223] Next, the structure of the optical system housed in the housing of the light source device 13 will be described in detail with reference to FIG. 6 and FIG. 7

[0224] Because FIG. 6 and FIG. 7 are cross-sectional views, only one of the plurality of LED elements 201 constituting the light source is shown, and they are transformed into substantially collimated light by the shape of the light-receiving end surface 203a of the light guide 203. Therefore, the light-receiving portion of the end surface of the light guide is installed in a predetermined positional relationship with the LED element.

[0225] In addition, the light guide 203 is formed of, for example, a light-transmissive resin such as an acrylic resin. Furthermore, the LED light-receiving surface of the end portion of the light guide 203 has, for example, a convex conical shape outer peripheral surface obtained by rotating a parabolic cross section, a top portion having a recess, the recess having a convex portion (i.e., a convex lens surface) formed in the central portion thereof, and a convex lens surface (or a concave lens surface that can be recessed inward) protruding outward in the central portion of the planar portion (not shown). In addition, the light-receiving portion of the light guide in which the LED element 201 is installed has a parabolic shape that forms a conical shape outer peripheral surface, is set within an angle range in which light emitted in the peripheral direction from the LED element is totally reflected inside, or a reflection surface is formed.

[0226] ​On the other hand, the LED elements 201 are arranged at predetermined positions on the surface of the circuit board, i.e., the LED substrate 202. The LED substrate 202 is arranged and fixed so that the LED elements 201 on the surface thereof are respectively positioned at the central portions of the recesses with respect to the light-receiving end surface 203a of the light guide 203.

[0227] According to this structure, the light emitted from the LED elements 201 can be made into substantially parallel light by the shape of the light-receiving end surface 203a of the light guide 203 and output, and the utilization efficiency of the generated light can be improved.

[0228] As described above, the light source device 13 is configured by mounting the light source unit in which a plurality of LED elements 201 as light sources are arranged at the light-receiving portion, i.e., the light-receiving end surface 203a provided at the end surface of the light guide 203. For the divergent light beams from the LED elements 201, the lens shape of the light-receiving end surface 203a of the light guide end surface is used to make them into substantially parallel light, and the light is guided inside the light guide 203 as indicated by arrows (in the direction parallel to the paper surface), and the light beams are made to be emitted toward the liquid crystal display panel 11 (in the direction perpendicular to the paper surface and outward) arranged substantially in parallel to the light guide 203 by the light beam direction conversion unit 204. By optimizing the distribution (density) of the light beam direction conversion unit 204 by the shape of the inside or surface of the light guide, the uniformity of the light beams incident on the liquid crystal display panel 11 can be controlled.

[0229] The light beam direction conversion unit 204 described above makes the light beams propagating inside the light guide to be emitted toward the liquid crystal display panel 11 (in the direction perpendicular to the paper surface and outward) arranged substantially in parallel to the light guide 203 by using the shape of the surface of the light guide or providing a portion having different refractive indexes inside the light guide. At this time, for the liquid crystal display panel 11, in a state where the center of the screen is directly opposite and the viewpoint is positioned at the same position as the diagonal dimension of the screen, the luminance of the central portion of the screen is compared with that of the peripheral portion of the screen. As long as the relative luminance ratio is 20% or more, it is not a problem in practice, and if it exceeds 30%, the characteristics are more excellent.

[0230] In addition, FIG. 6 is a cross-sectional configuration diagram for explaining the structure of the light source of the present embodiment in which the polarization conversion is performed in the light source device 13 including the light guide 203 and the LED elements 201 and the function thereof. FIG. 6 In the light source device 13, for example, the light guide 203 in which the light beam direction conversion unit 204 is provided on the surface or inside thereof is formed of plastic or the like, the LED elements 201 as light sources, a reflection sheet 205, a phase difference plate 206, a cylindrical lens, or the like, and on the upper surface thereof, the liquid crystal display panel 11 in which a polarizing plate is provided on the light source light incident surface and the image light emission surface is mounted.

[0231] In addition, a thin film or sheet-like reflective polarizer 49 is provided on the light source light incident surface (the lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13, so that one polarization (for example, P light) 212 in the natural light beam 210 emitted from the LED element 201 is selectively reflected. The reflected light is reflected again on the reflective plate 205 provided on one surface (the lower surface in the figure) of the light guide 203 and goes to the liquid crystal display panel 11. Therefore, a phase difference plate (λ / 4 wave plate) is provided between the reflective plate 205 and the light guide 203 or between the light guide 203 and the reflective polarizer 49, so that the light is reflected on the reflective plate 205 and passes through the phase difference plate twice, thereby converting the reflected light beam from P polarization to S polarization, thereby improving the utilization efficiency of the light source light as the image light. The image light beam ( FIG. 6 The light is incident on the retro-reflective plate 2 as indicated by the arrow 213. After being reflected on the retro-reflective plate 2, a real image suspended in space can be obtained.

[0232] FIG. 7 is with FIG. 6 Similarly, a cross-sectional configuration diagram is provided for explaining the structure and function of the light source of this embodiment, which performs polarization conversion in light source device 13, which includes a light guide 203 and an LED element 201. Light source device 13 also includes, for example, a light guide 203 formed of plastic or the like, with a beam direction conversion unit 204 provided on or within the light source, the LED element 201 serving as the light source, a reflective sheet 205, a phase difference plate 206, a cylindrical lens, and the like. A liquid crystal display panel 11, having polarizing plates on both the light source light incident surface and the image light exit surface, is mounted on the upper surface of light source device 13 as an image display element.

[0233] A reflective polarizing plate 49 in the form of a film or sheet is provided on the light source incident surface (the lower surface in the figure) of the liquid crystal display panel 11 corresponding to the light source device 13 to selectively reflect one polarization (e.g., S light) 211 of the natural light beam 210 emitted from the LED element 201. FIG. 7 In the example, the selective reflection characteristics of the reflective polarizer 49 are FIG. 7 Different. The reflected light is reflected by the reflective sheet 205 provided on one surface of the light guide 203 (the lower side in the figure) and goes to the liquid crystal display panel 11 again. A phase difference plate (λ / 4 wave plate) is provided between the reflective sheet 205 and the light guide 203 or between the light guide 203 and the reflective polarizer 49, so that the light is reflected by the reflective sheet 205 and passes through the phase difference plate twice, thereby converting the reflected light beam from S polarization to P polarization, thereby improving the utilization efficiency of the light source light as the image light. The image light beam ( FIG. 6 The light is incident on the retro-reflective plate 2 as indicated by the arrow 214. After being reflected on the retro-reflective plate 2, a real image suspended in space can be obtained.

[0234] In FIG. 7 and FIG. 8 the light source device shown in FIG. 1, in addition to the effect of the polarizing plate provided on the light incident surface of the corresponding liquid crystal display panel 11, since one polarization component is reflected by the reflective polarizing plate, the contrast ratio that can be obtained in theory is the product of the reciprocal of the orthogonal transmittance of the reflective polarizing plate and the reciprocal of the orthogonal transmittance obtained from the two polarizing plates attached to the liquid crystal display panel. Thus, a higher contrast ratio performance can be obtained. In fact, it was confirmed through experiments that the contrast ratio performance of the display image was improved by more than 10 times. As a result, a high-quality image comparable to that of the self-emitting type organic EL can be obtained.

[0235] Example 2 of the display device

[0236] FIG. 9 Another example of the specific structure of the display device 1 is shown. The light source device 13 is constructed, for example, by housing an LED, a collimator, a synthetic diffusion block, a light guide, and the like in a case of plastic or the like, and mounting the liquid crystal display panel 11 on the upper surface thereof. In addition, an LED substrate on which LED (Light Emitting Diode) elements 14a, 14b as semiconductor light sources and control circuits thereof are mounted is mounted on one side surface of the case of the light source device 13, and a heat sink 103 as a member for cooling heat generated in the LED elements and the control circuits is mounted on the outer side surface of the LED substrate.

[0237] In addition, on the liquid crystal display panel frame mounted on the upper surface of the case, the liquid crystal display panel 11 mounted on the frame, an FPC (Flexible Printed Circuits) 403 electrically connected to the liquid crystal display panel 11, and the like are provided. That is, the liquid crystal display panel 11 as a liquid crystal display element and the LED elements 14a, 14b as solid light sources generate a display image by modulating the intensity of transmitted light based on a control signal from a control circuit (not shown here) constituting an electronic device.

[0238] Example 3 of the display device

[0239] Next, using FIG. 9Other example illustrating the detailed configuration of the display device 1 (Example 3 of the display device). The light source device of this display device 1 converts the divergent light beam of the light (P-polarized light and S-polarized light mixed) from the LED into a substantially parallel light beam using the collimator 18, and reflects it toward the liquid crystal display panel 11 using the reflecting surface of the reflective light guide 304. The reflected light is incident on the reflective polarizing plate 49 disposed between the liquid crystal display panel 11 and the reflective light guide 304. The reflective polarizing plate 49 transmits light of a specific polarization (e.g., P-polarized light), and the transmitted polarized light is incident on the liquid crystal display panel 11. Here, light of other polarizations (e.g., S-polarized light) other than the specific polarization is reflected on the reflective polarizing plate 49, and goes back to the reflective light guide 304 again.

[0240] The reflective polarizing plate 49 is disposed inclined with respect to the liquid crystal display panel 11 in a manner that the chief ray of the light from the reflecting surface of the reflective light guide 304 is not perpendicular. The chief ray of the light reflected on the reflective polarizing plate 49 is incident on the transmission surface of the reflective light guide 304. The light incident on the transmission surface of the reflective light guide 304 transmits through the back surface of the reflective light guide 304, and is reflected on the reflecting plate 271 through the λ / 4 wave plate 270 as a phase difference plate. The light reflected on the reflecting plate 271 transmits through the λ / 4 wave plate 270 again, and transmits through the transmission surface of the reflective light guide 304. The light transmitted through the transmission surface of the reflective light guide 304 is incident on the reflective polarizing plate 49 again.

[0241] At this time, the light incident on the reflective polarizing plate 49 again has passed through the λ / 4 wave plate 270 twice, and thus the polarization is converted to a polarization (e.g., P-polarization) that can transmit through the reflective polarizing plate 49. Thus, the light after the polarization conversion transmits through the reflective polarizing plate 49 and is incident on the liquid crystal display panel 11. Note that the polarizations in the polarization conversion can be reversed (S-polarization and P-polarization are exchanged) compared with the above description.

[0242] As a result, the light from the LED is unified to a specific polarization (e.g., P-polarization) and is incident on the liquid crystal display panel 11, and the image is displayed on the panel surface by the luminance modulation corresponding to the image signal. As in the above example, a plurality of LEDs (only one is illustrated in the figure because it is a longitudinal sectional view) constituting the light source are mounted at prescribed positions with respect to the collimator 18. FIG. 9

[0243] ​Further, the collimator 18 is formed of a light-transmissive resin such as an acrylic resin or glass, for example. The collimator 18 can have a convex conical outer surface obtained by rotating a parabolic cross section. Further, in the central portion of the top portion (the side opposite to the LED substrate 102) of the collimator 18, a concave portion having a convex lens surface formed therein can be provided. Further, in the central portion of the flat portion (the side opposite to the top portion) of the collimator 18, a convex lens surface protruding outward (or a concave lens surface recessed inward) can be provided. Further, the parabolic surface of the conical outer surface of the collimator 18 is set within an angle range in which light emitted from the LED toward the peripheral direction is totally reflected inside, or a reflecting surface is formed.

[0244] Further, the LEDs are arranged at predetermined positions on the surface of the circuit board, i.e., the LED substrate 102, respectively. The LED substrate 102 is arranged and fixed so that the LEDs on the surface thereof are positioned at the central portion of the top portion of the convex conical shape (or the concave portion in the case where the top portion has the concave portion) of the collimator 18, respectively.

[0245] According to this structure, under the action of the collimator 18, light emitted from the LEDs, particularly light emitted from the central portions thereof, is converged into parallel light by the convex lens surface of the outer shape of the collimator 18. Further, light emitted toward the peripheral direction from the other portions is reflected by the parabolic surface of the conical outer surface of the collimator 18, and is also converged into parallel light. In other words, the collimator 18 in which the central portions constitute convex lenses and the peripheral portions have parabolic surfaces can make almost all of the light generated by the LEDs into parallel light output, and thus can improve the utilization efficiency of the generated light.

[0246] Further, FIG. 9The light converted into substantially parallel light by the collimator 18 is reflected by the reflective light guide 304. Of this light, light of a specific polarization is transmitted through the reflective polarizer 49 by the action of the reflective polarizer 49. Light of a different polarization, reflected by the reflective polarizer 49, is then transmitted again through the light guide 304. This light is reflected by the reflective plate 271, which is located opposite the liquid crystal display panel 11 relative to the reflective light guide 304. At this point, the light undergoes polarization conversion by passing twice through the λ / 4 wave plate 270, which functions as a phase difference plate. The light reflected by the reflective plate 271 is then transmitted again through the light guide 304 and enters the reflective polarizer 49, located on the opposite side. Because this incident light has undergone polarization conversion, it can pass through the reflective polarizer 49 and enter the liquid crystal display panel 11 with a uniform polarization direction. As a result, all the light from the light source can be utilized, doubling the geometrical efficiency of light. Furthermore, the polarization degree (extinction ratio) of the reflective polarizer is multiplied by the overall system extinction ratio. Therefore, using the light source device of this embodiment significantly improves the overall contrast of the display device. Furthermore, by adjusting the surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflective plate 271, the reflected diffusion angle of light on each reflective surface can be adjusted. To achieve greater uniformity of light incident on the liquid crystal display panel 11, the surface roughness of the reflective surface of the reflective light guide 304 and the surface roughness of the reflective plate 271 can be adjusted for each design.

[0247] in addition, FIG. 9 The phase difference plate, that is, the λ / 4 wave plate 270, does not need to have a phase difference of λ / 4 for polarized light incident perpendicularly to the λ / 4 wave plate 270. FIG. 10 In this structure, a retardation plate that changes the phase of polarized light by 90° (λ / 2) after passing through it twice can be used. The thickness of the retardation plate can be adjusted according to the incident angle distribution of the polarized light.

[0248] <Example 4 of Display Device>

[0249] Furthermore, use FIG. 10 Another example of the structure of the optical system, such as the light source device, of a display device (Display Device Example 4) will be described. This is a structural example in which a diffuser sheet is used in place of the reflective light guide 304 in the light source device of Display Device Example 3. Specifically, two optical sheets (optical sheet 207A and optical sheet 207B) are used on the light-emitting side of the collimator 18 to transform the diffusion characteristics in the vertical and horizontal directions (front-back direction in the figure, not shown). Light from the collimator 18 is incident between the two optical sheets (diffuser sheets).

[0250] In addition, the optical sheet can be replaced by a single sheet instead of a two-sheet structure. In the case of a single-sheet structure, the vertical and horizontal diffusion characteristics can be adjusted by the fine shapes of the front and back of the single optical sheet. In addition, multiple diffusion sheets can also be used to share the role. Here, FIG. 10 In this example, the reflective diffusion characteristics determined by the front and back shapes of the optical sheets 207A and 207B can be optimized using the number of LEDs, the divergence angle of the LED substrate (optical element) 102, and the optical specifications of the collimator 18 as design parameters to achieve a uniform surface density of the light beams emitted from the liquid crystal display panel 11. In other words, the diffusion characteristics are adjusted using the surface shapes of multiple diffusion sheets instead of light guides.

[0251] FIG. 10 In the example of , polarization conversion is performed in the same manner as in the example 3 of the above-mentioned display device. FIG. 10 In the example of FIG. 1 , the reflective polarizing film 49 can be configured to have a characteristic of reflecting S-polarized light (transmitting P-polarized light). In this case, the P-polarized light emitted from the light source, the LED, is transmitted, and the transmitted light is incident on the liquid crystal display panel 11. The S-polarized light emitted from the light source, the LED, is reflected, and the reflected light passes through the liquid crystal display panel 11. FIG. 10 The light passing through the phase difference plate 270 is reflected by the reflector 271. The light reflected by the reflector 271 is again converted to P-polarized light by the phase difference plate 270. The polarization-converted light passes through the reflective polarizer 49 and enters the liquid crystal display panel 11.

[0252] in addition, FIG. 10 The phase difference plate, that is, the λ / 4 wave plate 270, does not need to have a phase difference of λ / 4 for polarized light incident perpendicularly to the λ / 4 wave plate 270. FIG. 10 In the structure, a phase difference plate can be used as long as the polarized light passes through twice and the phase is changed by 90° (λ / 2). The thickness of the phase difference plate can be adjusted accordingly according to the incident angle distribution of the polarized light. In addition, FIG. 12 The same is true in the case of polarization conversion. The polarization design in the polarization conversion can make the polarization opposite (swap S polarization and P polarization) compared to the above description.

[0253] Regarding the light emitted from the liquid crystal display panel 11, in a device for general TV use, the light emitted in the horizontal direction of the screen (indicated by FIG. 12 (a) X-axis) and the vertical direction of the screen (with FIG. 12 The Y axis of (b) shows the same diffusion characteristics. In contrast, the diffusion characteristics of the light beam emitted from the liquid crystal display panel of this embodiment are as follows: FIG. 12As shown in Example 1, the viewing angle at 50% of the brightness of the front view (0 degrees) is 13 degrees, which is about 1 / 5 of the 62 degrees of the device for the usual TV use. Also, for the viewing angle in the vertical direction, the reflection angle and the area of the reflection surface of the reflection-type light guide are optimized in such a manner that the viewing angle is made uniform from top to bottom and is suppressed to about 1 / 3 of the viewing angle of the lower side. As a result, compared with the existing liquid crystal TV, the amount of image light going to the viewing direction is greatly increased, and the brightness is increased by 50 times or more.

[0254] Further, if the viewing angle characteristics shown in Example 2 are adopted, FIG. 11 the viewing angle at 50% of the brightness of the front view (0 degrees) is 5 degrees, which is 1 / 12 of the 62 degrees of the device for the usual TV use. Also, for the viewing angle in the vertical direction, the reflection angle and the area of the reflection surface of the reflection-type light guide are optimized in such a manner that the viewing angle is made uniform from top to bottom and is suppressed to about 1 / 12 of the viewing angle of the device for the usual TV use. As a result, compared with the existing liquid crystal TV, the amount of image light going to the viewing direction is greatly increased, and the brightness is increased by 100 times or more.

[0255] By making the viewing angle narrow as described above, the amount of light beam going to the viewing direction can be concentrated, so the light utilization efficiency is greatly increased. As a result, even if the liquid crystal display panel for the usual TV use is used, a great increase in brightness can be achieved at the same power consumption by controlling the light diffusion characteristics of the light source device, and an image display device corresponding to the information display system for bright outdoor use can be realized.

[0256] In the case where a large liquid crystal display panel is used, the light at the periphery of the screen is directed inward so as to go to the viewer in the case where the viewer is directly facing the center of the screen, and thus the uniformity of the screen brightness is improved. FIG. 9 The convergence angle of the long side and the short side of the panel is found when the distance L of the viewer from the panel and the size of the panel (picture ratio 16:10) are parameters. In the case of vertical screen viewing, the convergence angle is set in correspondence with the short side, for example, in the case where a 22" panel is used in vertical screen and the viewing distance is 0.8 m, if the convergence angle is set to 10 degrees, the image light from the four corners of the screen can be effectively made to go to the viewer.

[0257] Also, in the case of vertical screen viewing using a 15" panel, in the case where the viewing distance is 0.8 m, if the convergence angle is set to 7 degrees, the image light from the four corners of the screen can be effectively made to go to the viewer. As described above, by making the image light at the periphery of the screen go to the viewer who is located at the position most suitable for viewing the center of the screen in accordance with the size of the liquid crystal display panel and whether the vertical screen use or the horizontal screen use, the uniformity of the screen brightness can be improved.

[0258] As the basic structure, as described aboveFIG. 13A As shown, a light source device is used to make a light beam with narrow-angle directional characteristics incident on the liquid crystal display panel 11, and the brightness is modulated accordingly according to the image signal, so that the image information displayed on the screen of the liquid crystal display panel 11 is reflected on the retroreflective plate to obtain a spatial floating image, which is displayed outdoors or indoors through the transparent component 100.

[0259] By using the display device and the light source device according to one embodiment of the present invention described above, a spatial floating image display device with higher light utilization efficiency can be realized.

[0260] <Example of Image Display Processing in a Spatial Floating Image Display Device>

[0261] Next, for an example of the technical problem to be solved by the image processing of this embodiment, use FIG. 13A In the spatial floating image display device 1000 , the back side of the spatial floating image 3 is located inside the housing of the spatial floating image display device 1000 as viewed by the user. In sufficiently dark conditions, the user sees the background of the spatial floating image 3 as black.

[0262] Here, use FIG. 3 An example of displaying the character "Panda" 1525 in the space floating image 3 is described. First, FIG. 13A The image control unit 1160 is for FIG. 13A The image shown in (1) - which includes a pixel area of ​​an image depicting a character "panda" 1525 and a transparent information area 1520 as a background image, distinguishes the pixel area of ​​the image depicting the character "panda" 1525 from the transparent information area 1520 as the background image.

[0263] Regarding the method for distinguishing a character image from a background image, for example, the image processing of the image control unit 1160 may be configured to process the background image layer and the layer of the character image located in front of the background image layer as different layers, and distinguish the character image from the background image based on the overlapping relationship when these layers are synthesized.

[0264] Here, the image control unit 1160 recognizes the black color of the pixels depicting an object such as a character image as information different from the transparent information pixels. However, the pixel brightness of both the black pixels depicting the object and the transparent information pixels is 0. In this case, when the spatial floating image 3 is displayed, there is no brightness difference between the black pixels in the image depicting the character "panda" 1525 and the pixels in the transparent information area 1520 as the background image. Therefore, in the spatial floating image 3, as shown in FIG. FIG. 13BIn (2) of FIG. 15, the pixels of the image of the character "Panda" 1525 that are black and the pixels of the transparent information area 1520 are all without luminance, and are recognized by the user as optically the same black space. That is, the black portion of the image of the object, the character "Panda" 1525, is integrated with the background, and only the non-black portion of the character "Panda" 1525 can be recognized as an image suspended in space in the display area of the spatially suspended image 3.

[0265] For an example of image processing of the present embodiment, the image processing of the image processing unit 1150 of the display device 1 shown in FIG. 15 is used. FIG. 13B An example of image processing that can better solve the technical problem of the black image area of the object (object) being integrated with the background described in the related art is explained with reference to FIG. 15. FIG. 13A In (1) and (2) of FIG. 15, the display state of the spatially suspended image 3 is shown on the upper side, and the input-output characteristics of the image processing of the image of the object are shown on the lower side. In addition, the image of the object (the character "Panda" 1525) and / or data corresponding thereto can be read from the storage section 1170 or the memory 1109 of the display device 1. Alternatively, it can be input from the image signal input section 1131. Alternatively, it can be acquired via the communication section 1132. FIG. 13B FIG. 3 Here, in the state of (1) of FIG. 15, the input-output characteristics of the image processing of the image of the object are in a state of linear without special adjustment. In this case, it is the same display state as (2) of FIG. 15, and the black image area of the object is integrated with the background. In contrast, in (2) of FIG. 15, the image control section 1160 of the present embodiment adjusts the input-output characteristics of the image processing of the image of the object (the character "Panda" 1525) to the input-output characteristics shown on the lower side. FIG. 13B That is, the image control section 1160 performs image processing on the image of the object (the character "Panda" 1525) with input-output characteristics in which, for the pixels of the input image, the luminance value of the pixels of the low luminance area is increased to output pixels. After the image processing of the image of the object (the character "Panda" 1525) is performed with the input-output characteristics, the image including the image of the object (the character "Panda" 1525) is input to the display device 1 to be displayed. In this way, the display state of the spatially suspended image 3 is as shown in (2) of FIG. 15.

[0266] FIG. 13A FIG. 13B FIG. 13B

[0267] FIG. 13B ​​​​​​As shown in the upper portion of (2), the brightness of the black pixel area in the image depicting the character "Panda" 1525 is increased. As a result, the black area in the image depicting the character "Panda" 1525 is not blended in with the black background, allowing the user to distinguish and recognize it, thereby better displaying the object.

[0268] That is, by using FIG. 3 The image processing of (2) shows the area where the image of the object, i.e., the character "panda" 1525, is displayed, and can be distinguished from the black background of the interior of the housing of the spatial floating image display device 1000, i.e., the interior of the housing, which is visible through the window, thereby improving the recognizability of the object. FIG. 13B When the image of the above-mentioned object and / or the corresponding data are read from the storage unit 1170 or the memory 1109, or when the image of the above-mentioned object is input from the image signal input unit 1131, or when the data of the above-mentioned object is obtained via the communication unit 1132, etc.), the object whose pixels constituting the object include pixels with a brightness value of 0 can also be converted into an object in which the brightness values ​​of pixels in the low-brightness area are increased through the image processing of the input-output characteristics performed by the image control unit 1160, and then displayed on the display device 1, and converted into the spatial floating image 3 by the optical system of the spatial floating image display device 1000.

[0269] That is, the pixels of the image processed object constituting the input-output characteristic do not include pixels with a brightness value of 0. After being transformed into such a state, the image is displayed on the display device 1 and transformed into the spatial floating image 3 by the optical system of the spatial floating image display device 1000.

[0270] In addition, FIG. 13B In the image processing of (2), the image processing is performed only on the region of the image of the object (character "panda" 1525). FIG. 13B The image processing method of the input-output characteristic of (2) is configured, for example, to process the background image layer and the character image layer located in front of the background image layer as different layers in the image processing of the image control unit 1160, and to perform FIG. 13B The image processing of the input and output characteristics of (2) is not performed on the background image layer.

[0271] These layers are then composited, so that FIG. 13B As shown in (2), only the character image is subjected to the image processing characteristic of brightening the low-brightness area of ​​the input image. In addition, as another method, it is also possible to construct a method in which, after the character image layer and the background image layer are synthesized, only the character image area is subjected to the image processing characteristic of brightening the low-brightness area of ​​the input image. FIG. 13B (2) Input-output characteristics of image processing.

[0272] In the image processing for brightening a low-brightness area in the input-output characteristic of an input image, the input-output image characteristic used is not limited to FIG. 13B (2) is an example. Any image processing that can brighten low brightness can be used, such as so-called brightness adjustment. Alternatively, image processing such as that disclosed in International Publication No. 2014 / 162533 can be performed to improve visibility by controlling the gain used to change the weight of the Retinex process.

[0273] According to the above description FIG. 13A The image processing (2) can prevent the black area in the area of ​​the image depicting characters, objects, etc. from blending in with the black background, making it easier for the user to recognize and achieving better display.

[0274] In addition, FIG. 13B 、 FIG. 4A-4G In the example, a space-suspended image display device (e.g. FIG. 4I The spatial suspended image display device 1000, FIG. 4J 、 FIG. 4H The technical problems and better image processing are described using the spatial floating image display device 1000 with the back side window blocked as an example. However, this image processing is also effective in devices other than these spatial floating image display devices.

[0275] Specifically, in FIG. 4I The spatial suspended image display device 1000 and FIG. 4J 、 FIG. 13A In the spatial floating image display device 1000 in the state where the back side window is not shielded, the background of the spatial floating image 3 is not black, but the scenery behind the spatial floating image display device 1000 across the window. FIG. 13B and FIG. 13B The technical problems described also exist.

[0276] That is, the black portion of the image depicting the object character "panda" 1525 is integrated with the scenery behind the space floating image display device 1000 across the window. FIG. 13B The image processing (2) allows the black portion of the image depicting the object, ie, the character "panda" 1525, to be distinguished from the scenery behind the spatial floating image display device 1000 across the window, thereby improving the recognizability of the object.

[0277] That is, by using FIG. 4KThe image processing of (2) allows the area where the image of the object, i.e., the character "panda" 1525, is displayed to be distinguished from the scenery behind the spatial floating image display device 1000 across the window, and allows better recognition that the object, i.e., the character "panda" 1525, is located in front of the above scenery, thereby improving the recognizability of the object.

[0278] In addition, FIG. 4L 、 FIG. 4M 、 FIG. 13A In the spatial floating image display device 1000, as described above, when another image (such as the image of the transmissive self-luminous image display device 1650 or the image of the second display device 1680) is displayed at a depth different from that of the spatial floating image 3, the background of the spatial floating image 3 is not black but the other image. In this case, FIG. 13B and FIG. 13B The technical problems described also exist.

[0279] That is, the black portion of the image depicting the object character "panda" 1525 is integrated with the other images displayed at a different depth position from the spatial floating image 3. In this case, too, by using FIG. 13B The image processing (2) enables the black portion of the image depicting the object, ie, the character "panda" 1525, to be distinguished from the other images, thereby improving the recognizability of the object.

[0280] That is, by using FIG. 13C The image processing of (2) shows that the area of ​​the image of the object, i.e., the character "panda" 1525, can be distinguished from the above-mentioned other images, and it can be better recognized that the object, i.e., the character "panda" 1525, is located in front of the above-mentioned other images, and the recognizability of the object is improved.

[0281] For an example of image display processing in this embodiment, use FIG. 13C Provide explanation. FIG. 4K In the example of image display of this embodiment, a spatial floating image 3 and another image, namely a second image 2050, are displayed simultaneously. The second image 2050 may correspond to FIG. 4L or FIG. 4M The second image 2050 may also correspond to the display image of the transmissive self-luminous image display device 1650. FIG. 13C The display image of the second display device 1680.

[0282] Right now, FIG. 4K An example of an image display FIG. 4L 、 FIG. 4M 、 FIG. 13CFig. 1 is an example of a specific example of the image display example of the spatially-suspended image display device 1000. In the example of this figure, a bear character is displayed in the spatially-suspended image 3. The area other than the bear character in the spatially-suspended image 3 is black, and the spatially-suspended image is transparent. In addition, the second image 2050 is a background image depicting a plain, a mountain, and the sun.

[0283] Here, FIG. 13D In this case, the spatially-suspended image 3 and the second image 2050 are displayed at different depth positions. The user 230 views the two images of the spatially-suspended image 3 and the second image 2050 in the line-of-sight direction of the arrow 2040, and thus the user 230 can view the images in a state in which the two images overlap. Specifically, in front of the background of the plain, the mountain, and the sun depicted in the second image 2050, the bear character of the spatially-suspended image 3 can be seen in an overlapping manner.

[0284] Here, the spatially-suspended image 3 is imaged as a real image in the air, and thus when the user 230 slightly moves the viewpoint, the user 230 can recognize the depth of the spatially-suspended image 3 and the second image 2050 due to parallax. Thus, the user 230 can view the two images in an overlapping state while obtaining a stronger sense of spatial suspension with respect to the spatially-suspended image 3.

[0285] An example of the image display processing of the present embodiment will be described using FIG. 13D . FIG. 13C (1) of Fig. 2 is FIG. 13D Fig. 2 is an example of the image display of the present embodiment, and is a view of the spatially-suspended image 3 from the line-of-sight direction of the user 230. In this case, a bear character is displayed in the spatially-suspended image 3. The area other than the bear character in the spatially-suspended image 3 is black, and the spatially-suspended image is transparent.

[0286] FIG. 13C (2) of Fig. 2 is FIG. 13D Fig. 2 is an example of the image display of the present embodiment, and is a view of the second image 2050 from the line-of-sight direction of the user 230. In the example of this figure, the second image 2050 is a background image depicting a plain, a mountain, and the sun.

[0287] FIG. 13C (3) of Fig. 2 is a view indicating FIG. 3 Fig. 2 is an example of the image display of the present embodiment, and is a view of the state in which the second image 2050 and the spatially-suspended image 3 are seen in an overlapping manner in the line-of-sight direction of the user 230. Specifically, in front of the background of the plain, the mountain, and the sun depicted in the second image 2050, the bear character of the spatially-suspended image 3 can be seen in an overlapping manner.

[0288] Here, in a case where the spatially-suspended image 3 and the second image 2050 are displayed at the same time, in order to more surely ensure the recognizability of the spatially-suspended image 3, it is preferable that the brightness of both images be kept in balance. If the second image 2050 is too bright compared to the brightness of the spatially-suspended image 3, the displayed image of the spatially-suspended image 3 will become transparent, and the background, that is, the second image 2050 will be clearly seen in perspective.

[0289] Therefore, the output of the light source of the spatially-suspended image 3 and the display image brightness of the display device 1, the output of the light source of the display device that displays the second image 2050, and the display image brightness of the display device can be set so that at least the brightness per unit area of the spatially-suspended image 3 at the display position of the spatially-suspended image 3 is greater than the brightness per unit area of the image light that reaches the display position of the spatially-suspended image 3 from the second image 2050.

[0290] In addition, as long as this condition is satisfied in a case where the spatially-suspended image 3 and the second image 2050 are displayed at the same time, in a case where the first display mode in which the spatially-suspended image 3 is not displayed and only the second image 2050 is displayed is switched to the second display mode in which the spatially-suspended image 3 and the second image 2050 are displayed at the same time, it is possible to control the brightness of the second image 2050 to be reduced by reducing the output of the light source of the display device that displays the second image 2050 and / or the display image brightness of the display device. These controls can be implemented by the control section 1110 controlling the display device 1 and the display device that displays the second image 2050 FIG. 4K the transmissive self-luminous image display device 1650 or FIG. 4L the second display device 1680) of the display device 1. FIG. 4M FIG. 14 In addition, in the switching from the above-described first display mode to the above-described second display mode, in a case where the brightness of the second image 2050 is to be reduced, the brightness can be reduced uniformly over the entire screen of the second image 2050. Alternatively, instead of reducing the brightness uniformly over the entire screen of the second image 2050, the portion of the spatially-suspended image 3 in which the object is to be displayed can be made to be in a state in which the brightness reduction effect is the highest, and the brightness reduction effect can be made to be alleviated in stages in the periphery thereof. This is because, as long as the brightness reduction of the second image 2050 is implemented only in the portion of the second image 2050 in which the spatially-suspended image 3 is seen in superposition, it is sufficient to ensure the recognizability of the spatially-suspended image 3.

[0291]

[0292] ​​Here, the spatially suspended image 3 and the second image 2050 are displayed at different depths. Therefore, if the user 230 slightly changes their viewpoint, the overlapping position of the spatially suspended image 3 on the second image 2050 will change due to parallax. Therefore, when switching from the first display mode to the second display mode, when the brightness of the entire second image 2050 is unevenly reduced, it is not preferable to sharply reduce the brightness based on the outline of the object displayed in the spatially suspended image 3. Instead, it is preferable to gradually change the brightness reduction effect according to the position as described above.

[0293] In addition, if the position of the object displayed in the spatial floating image 3 is approximately the center of the spatial floating image 3, then in the spatial floating image display device 1000, the position where the brightness reduction effect is highest in the gradual brightness reduction process can be set to the center of the spatial floating image 3.

[0294] According to the image display processing of this embodiment described above, the user 230 can better view the spatial floating image 3 and the second image 2050 .

[0295] Furthermore, the second image 2050 may be controlled not to be displayed when the spatial floating image 3 is displayed. When the second image 2050 is not displayed, the spatial floating image 3 is more recognizable, making it suitable for applications such as the spatial floating image display device 1000 where the user must reliably see the spatial floating image 3 when the spatial floating image 3 is displayed.

[0296] <Example 2>

[0297] As a second embodiment of the present invention, another example of the structure of the space floating image display device is described. In addition, the space floating image display device of this embodiment changes the optical system contained in the space floating image display device described in the first embodiment to FIG. 14 (1) or FIG. 14 The optical system shown in (2) is described in this embodiment. The differences from the first embodiment are described. For the same structure as the first embodiment, repeated description is omitted. In addition, in the description of this embodiment and the subsequent descriptions, it is specified that the polarization (polarized light) and the other polarization (polarized light) are polarizations (polarized light) with a phase difference of 90°.

[0298] FIG. 14 (1) is an example of the optical system and optical path of this embodiment. FIG. 2C The optical system shown in (1) FIG. 14 In the optical system, the display device 1 is brought closer to the polarization separation component 101B, making the entire optical system more compact. FIG. 2C (1) is marked with FIG. 14Structures of the same reference numerals are omitted from repeated detailed description.

[0299] FIG. 2C In (1) of the above-described embodiment, the display device 1, the polarization separation member 101B, and the retroreflective sheet 2 are arranged in a manner that the display device 1 is farther than the polarization separation member 101B and the retroreflective sheet 2. FIG. 2C Also, the image light of the prescribed polarization (P-polarization in the figure) that exits from the display device 1 travels in the vertical direction from the image display surface of the display device 1. Here, the polarization separation member 101B selectively transmits the prescribed polarization light (P-polarization in the figure) that exits from the display device 1 and reflects another polarization light (S-polarization in the figure). FIG. 14 Also, the image light of the prescribed polarization (P-polarization in the figure) that exits from the display device 1 travels in the vertical direction from the image display surface of the display device 1. Here, the polarization separation member 101B selectively transmits the prescribed polarization light (P-polarization in the figure) that exits from the display device 1 and reflects another polarization light (S-polarization in the figure).

[0300] Thus, the image light of the prescribed polarization (P-polarization in the figure) that travels in the vertical direction from the image display surface of the display device 1 transmits the polarization separation member 101B and reaches the retroreflective sheet 2 to which the λ / 4 wave plate 21 is attached. The image light that is retroreflective on the retroreflective sheet 2 and travels again toward the polarization separation member 101B transmits the λ / 4 wave plate 21 twice, and thus is converted from the prescribed polarization light (P-polarization in the figure) at the time of exit from the display device 1 to another polarization light (S-polarization in the figure). The image light that travels again toward the polarization separation member 101B is another polarization light (S-polarization in the figure), and thus is reflected on the polarization separation member 101B toward the position where the user should be located. The traveling direction of the image that is reflected on the polarization separation member 101B is determined based on the arrangement angle of the polarization separation member 101B.

[0301] FIG. 14 In the example of (1) of the above-described embodiment, the image light that travels toward the polarization separation member 101B is reflected on the polarization separation member 101B at right angles and travels as shown in the figure. The image light that is reflected on the polarization separation member 101B forms the spatially floating image 3A. The spatially floating image 3A can be well viewed by the user from the direction of the arrow A.

[0302] Here, there is a relationship in the characteristic of the retroreflectivity of the retroreflective sheet 2, in which the optical path length from the display device 1 to the retroreflective sheet 2 is equal to the optical path length from the retroreflective sheet 2 to the position where the spatially floating image 3A is formed. The position where the spatially floating image 3A is formed in the traveling direction of the image light that is reflected on the polarization separation member 101B is determined by this relationship.

[0303] FIG. 2C In the example of (1) of the above-described embodiment, the display device 1, the polarization separation member 101B, and the retroreflective sheet 2 are arranged in a manner that the display device 1 is farther than the polarization separation member 101B and the retroreflective sheet 2. FIG. 14 Thus, the image light of the prescribed polarization (P-polarization in the figure) that travels in the vertical direction from the image display surface of the display device 1 transmits the polarization separation member 101B and reaches the retroreflective sheet 2 to which the λ / 4 wave plate 21 is attached. The image light that is retroreflective on the retroreflective sheet 2 and travels again toward the polarization separation member 101B transmits the λ / 4 wave plate 21 twice, and thus is converted from the prescribed polarization light (P-polarization in the figure) at the time of exit from the display device 1 to another polarization light (S-polarization in the figure). The image light that travels again toward the polarization separation member 101B is another polarization light (S-polarization in the figure), and thus is reflected on the polarization separation member 101B toward the position where the user should be located. The traveling direction of the image that is reflected on the polarization separation member 101B is determined based on the arrangement angle of the polarization separation member 101B. FIG. 14The amount of floating of the optical system of (1) is not large. For example, as an indicator of the amount of floating of the spatial image 3A from the optical system, the distance from the position where the light of the central part of the image light is reflected on the polarization separation component 101B to the position where the spatial floating image 3A is formed by the image light is shown in the figure ( FIG. 14 In the example of (1) it is L1).

[0304] In addition, about FIG. 14 The polarization design in the optical system of (1) can also swap the characteristics of P polarization and S polarization. Specifically, the predetermined polarization of the image light emitted from the display device 1 can be made S polarization, and the reflection characteristics of the polarization separation component 101B can be swapped between P polarization and S polarization. In this case, the P polarization and S polarization shown in the figure are opposite, but the optical design of the optical path, etc. can be achieved in exactly the same way.

[0305] Then, in FIG. 14 Another example of the optical system and optical path of this embodiment is shown in (2). FIG. 14 The optical system of (2) is designed to achieve FIG. 14 The same compactness of the optical system of (1) increases the amount of floating images from the optical system, and changes FIG. 14 (1) structure in the optical system. FIG. 14 (2) is marked with FIG. 14 (1) The same reference numerals as those in the figure will not be described in detail.

[0306] FIG. 14 In (2), FIG. 14 (1) Similarly, the image light of a predetermined polarization (P polarization in the figure) emitted from the display device 1 travels in a vertical direction from the image display surface of the display device 1. Here, the polarization characteristics of the polarization separation component 101B are arranged in the same manner as FIG. 14 The image light of a predetermined polarization (P polarization in the figure) traveling in the vertical direction from the image display surface of the display device 1 passes through the polarization separation element 101B.

[0307] Here, with FIG. 14 Unlike (1), in front of the polarization separation element 101B, the image light is not provided with a retroreflective plate 2 having a λ / 4 wave plate 21 attached thereto, but with a specular reflective plate 4 having a λ / 4 wave plate 21B attached thereto. Here, the reflection on the specular reflective plate 4 is specular reflection (also called regular reflection), not retroreflection.

[0308] Thus, the image light that has passed through the polarization separation member 101B is specularly reflected on the mirror surface reflection plate 4 to which the λ / 4 wave plate 21B is attached. The image light that has been specularly reflected on the mirror surface reflection plate 4 and has traveled again toward the polarization separation member 101B passes through the λ / 4 wave plate 21 twice, and thus is converted from the prescribed polarized light (P-polarized light in the figure) at the time of ejection from the display device 1 to another kind of polarized light (S-polarized light in the figure). The image light that has traveled again toward the polarization separation member 101B is another kind of polarized light (S-polarized light in the figure), and thus is reflected on the polarization separation member 101B.

[0309] Here, FIG. 14 the direction of the configuration of the polarization separation member 101B in (2) is different from FIG. 14 (1), and thus the image light that is reflected on the polarization separation member 101B travels in the direction opposite to the position where the user should be located. In front of the traveling direction of the image light that is reflected on the polarization separation member 101B, the retroreflective plate 2 to which the λ / 4 wave plate 21C is attached is disposed. The image light is retroreflectively reflected by the retroreflective plate 2. The image light that has been retroreflectively reflected on the retroreflective plate 2 and has traveled again toward the polarization separation member 101B passes through the λ / 4 wave plate 21C twice, and thus is converted again from the other kind of polarized light (S-polarized light in the figure) to the prescribed polarized light (P-polarized light in the figure).

[0310] The image light that has traveled again toward the polarization separation member 101B is the prescribed polarized light (P-polarized light in the figure), and thus passes through the polarization separation member 101B and travels as it is toward the position where the user should be located. The image light that has passed through the polarization separation member 101B forms the spatially floating image 3B. The spatially floating image 3B can be favorably viewed by the user from the direction of the arrow A.

[0311] Here, FIG. 14 in (2), as well as FIG. 14 (1), there is such a relationship in the retroreflective characteristic of the retroreflective plate 2, in which the optical path length from the display device 1 to the retroreflective plate 2 of the image light is equal to the optical path length from the retroreflective plate 2 to the position at which the spatially floating image 3B is formed. The position at which the spatially floating image 3B is formed in the traveling direction of the image light that has passed through the polarization separation member 101B is determined by this relationship.

[0312] FIG. 14 The optical path length from the display device 1 to the retroreflective plate 2 of the image light in (2) is longer than FIG. 14 (1), because FIG. 14 In the optical system of (2), the optical path length from the display device 1 to the retroreflective plate 2 of the image light is longer than FIG. 14The optical path that goes back and forth between the polarization separation component 101B and the mirror reflection plate 4 does not exist in the optical system of (1).

[0313] thus, FIG. 14 The distance from the position where the central portion of the image light passes through the polarization separation component 101B to the position where the image light forms the spatially suspended image 3B in the optical system (2) is ( FIG. 14 In the example of (2), it is L2), and FIG. 14 The distance from the position where the light of the center portion of the image light is reflected on the polarization separation component 101B to the position where the image light forms the spatial suspended image 3A in the optical system (1) is ( FIG. 4E The example of (1) is very long compared to L1).

[0314] In addition, about FIG. 4F The polarization design in the optical system of (2) can also swap the characteristics of P polarization and S polarization. Specifically, the predetermined polarization of the image light emitted from the display device 1 can be made S polarization, and the reflection characteristics of the polarization separation component 101B can be swapped between P polarization and S polarization. In this case, the P polarization and S polarization shown in the figure are opposite, but the optical design of the optical path, etc. can be achieved in exactly the same way.

[0315] According to the embodiment 2 of the present invention described above FIG. 4G (1) and FIG. 4H The optical system of (2) can realize a more compact optical system. In particular, according to FIG. 4I The optical system of (2) can make the amount of the spatial suspended image floating from the optical system larger with a more compact optical system.

[0316] In addition, FIG. 4J (1) or FIG. 4K When the optical system of (2) is installed in the space floating image display device, the optical system of the space floating image display device described in Example 1 can be replaced by FIG. 4L (1) or FIG. 4M The optical system of (2) can be realized. Specifically, FIG. 14 The optical system of (1) is FIG. 4E 、 FIG. 4F 、 FIG. 4G 、 FIG. 4K 、 FIG. 4L 、 FIG. 15A 、 FIG. 15A 、 FIG. 15A or FIG. 3 In this case, since the optical system becomes compact, the housing of the spatial floating image display device in each figure can be made smaller.

[0317] In addition, specifically, the optical system of (2) of FIG. 15A may be replaced with the optical system of the spatially-suspended image display device of FIG. 3 , FIG. 15A , FIG. 3 , FIG. 15A or FIG. 15B . In this case, the amount by which the spatially-suspended image floats from the optical system can be made larger. In addition, since the optical system becomes compact, the housing of each of the spatially-suspended image display devices can be made smaller.

[0318] Embodiment 3

[0319] As Embodiment 3 of the present application, an example of a display example in the spatially-suspended image display device 1000 described in each of Embodiments 1 and 2 is described. As the spatially-suspended image display device 1000 of Embodiment 3 of the present application, any one of the spatially-suspended image display devices 1000 of each of Embodiments 1 and 2 can be used. In the present embodiment, only the points different from Embodiments 1 and 2 are described, and the description of the same structure as in these embodiments is omitted.

[0320] In the description of FIG. 15B , as an example of a display example in the spatially-suspended image display device 1000 of Embodiment 3, an example in which a rendered image of a 3D model of a character 1532 is displayed in the spatially-suspended image 3 is shown. The 3D model of the character 1532 displayed in the spatially-suspended image 3 is rendered as an image captured on a virtual 3D space from a prescribed viewpoint (virtual 3D space camera). The rendered image is displayed on the display device 1 and displayed in the spatially-suspended image 3 through the optical system of the spatially-suspended image display device 1000.

[0321] In the description of FIG. 15B , black display indicates a state in which black is displayed in the liquid crystal display panel 11 of the display device 1. The black display in the spatially-suspended image 3 is transparent in space, and thus there is no luminance and color, and it is empty space. That is, FIG. 15B In the example of

[0322] Using the structure of the spatially-suspended image display device 1000 of FIG. 15B , a display example in the spatially-suspended image display device 1000 for realizing, for example, FIG. 15C15. A first processing example of displaying a rendered image of a 3D model of a character 1532 or the like is shown. Specifically, an image generation program capable of generating a rendered image of a 3D model of a character or the like is first stored in storage unit 1170. Control unit 1110 reads the image generation program from storage unit 1170 and loads it into memory 1109. Control unit 1110 executes the image generation program loaded into memory 1109. The image generation program renders the 3D model of the character or the like to generate an image to be displayed on display device 1.

[0323] Here, the image control unit 1160 may control the display device 1 to display the generated image of the character 1532 .

[0324] In addition, use FIG. 15C The structure of the spatial suspended image display device 1000 is described, which is used to implement the following in the spatial suspended image display device 1000: FIG. 15C As shown, a second processing example of displaying a rendered image of a 3D model of a character 1532 or the like is shown.

[0325] In the second processing example, a pre-rendered image is stored (accumulated) in the storage unit 1170. In this example, the image control unit 1160 controls the reproduction (playback) of the rendered image stored in the storage unit 1170 and displays it on the display device 11. The pre-rendered image stored in the storage unit 1170 may be an image rendered by an image generation program loaded into the memory 1109 as in the first processing example.

[0326] In addition, the rendered image pre-stored in the storage unit 1170 may be pre-rendered by an external device, obtained by the spatial floating image display device 1000 via the communication unit 1132 , and stored in the storage unit 1170 .

[0327] In addition, use FIG. 15C The structure of the spatial suspended image display device 1000 is described, which is used to implement the following in the spatial suspended image display device 1000: FIG. 15C As shown in FIG. 3 , a third processing example is shown for displaying a rendered image of a 3D model of a character 1532, etc. In the third processing example, the spatial floating image display device 1000 obtains a rendered image pre-rendered by an external device via the communication unit 1132. In the third processing example, the rendered image obtained via the communication unit 1132 is not stored in the storage unit 1170, but is instead displayed on the display device 1 as a generated rendered image of the character 1532 under the control of the image control unit 1160.

[0328] Then, in FIG. 15DIn the description of Embodiment 3, as an example of the display example in the spatially-suspended image display apparatus 1000, an example of a rendered image in which an image of a virtual 3D space in which a 3D model of a character 1531 and a 3D model of a character 1532 are displayed in the spatially-suspended image 3 is shown. The 3D model of the character 1531 and the 3D model of the character 1532 displayed in the spatially-suspended image 3 are rendered as an image taken from a prescribed viewpoint on a virtual 3D space. The rendered image is displayed on the display apparatus 1 and displayed in the spatially-suspended image 3 through the optical system of the spatially-suspended image display apparatus 1000. In FIG. 15C In the description of Embodiment 3, black display indicates a state in which black is displayed in the liquid crystal display panel 11 of the display apparatus 1. Black display in the spatially-suspended image 3 is transparent in space, so there is no luminance and color, and it is empty space. That is, FIG. 15D In the example of Embodiment 3, only the character 1531 and the character 1532 displayed in the spatially-suspended image 3 are seen by the user in a manner of floating in the empty space.

[0329] Here, FIG. 15C is an example in which rendering is performed by parallel projection in the virtual 3D space in which the 3D model of the character 1531 and the 3D model of the character 1532 are present. Therefore, the height settings of the 3D model of the character 1531 and the 3D model of the character 1532 can be correctly recognized based on the rendered image. FIG. 16A is an example in which rendering is performed by parallel projection in the virtual 3D space in which the 3D model of the character 1531 and the 3D model of the character 1532 are present. Therefore, the height settings of the 3D model of the character 1531 and the 3D model of the character 1532 can be correctly recognized based on the rendered image.

[0330] Next, FIG. 16A is an example in which rendering is performed by perspective projection in the virtual 3D space in which the 3D model of the character 1531 and the 3D model of the character 1532 are present, and the rendered image is displayed on the display apparatus 1. Here, FIG. 16B is an example in which the field of view of the virtual 3D space camera serving as the rendering viewpoint is set to be wide-angle. Due to the wide-angle distortion of the virtual 3D space camera, the character 1532 standing near the periphery of the field of view appears to be more distorted than the character 1531 standing near the center of the field of view. Therefore, the character 1532 originally set to be shorter than the character 1531 appears to be taller than the character 1531.

[0331] For example, in a case in which a virtual 3D space in which a 3D model of a character 1531 and a 3D model of a character 1532 are displayed in the spatially-suspended image 3 is not displayed using the spatially-suspended image but is displayed on a fixed pixel display apparatus having a rectangular display region as usual, FIG. 16BIn the case of the same image, the area corresponding to the black area of ​​the spatially suspended image 3 is displayed in black on the fixed-pixel display, which the user can see. Therefore, the user can identify the rectangular display area of ​​the fixed-pixel display device as corresponding to the rectangle of the spatially suspended image 3. If the user can identify the rectangular display area of ​​the fixed-pixel display device, the user can infer the influence of wide-angle distortion based on this rectangle and the corresponding standing positions of characters 1532 and 1531, and the user can also recognize that character 1531 has been significantly deformed.

[0332] In contrast, when using space-suspended image display FIG. 15A In the case of the image, the black area of ​​the space floating image 3 is transparent to the user and cannot be seen. Therefore, the user cannot recognize the rectangular shape of the outer periphery of the display area of ​​the space floating image 3. FIG. 16B In the example shown in FIG. 3 , the user can only see characters 1531 and 1532 displayed in the spatially suspended image 3, floating in the air. The user recognizes that character 1532 is displayed larger than character 1531. Here, because the user cannot recognize the rectangular shape of the outer perimeter of the display area of ​​the spatially suspended image 3, it is difficult for the user to infer that the larger appearance of character 1532 is due to wide-angle distortion.

[0333] For ease of explanation, FIG. 16B Shown in FIG. 16B An example of displaying a black area within the rectangular display area of ​​the spatial floating image 3 in the image is eliminated.

[0334] Watch FIG. 16B As can be seen from the example of FIG. 3 The black area within the rectangular display area of ​​spatially suspended image 3 is eliminated from the image, making it difficult for the user to identify the center of the rectangular display area of ​​spatially suspended image 3. Consequently, it is difficult for the user to recognize that character 1531 is standing in the center of the rectangular display area of ​​spatially suspended image 3 and that character 1532 is standing at the right end of the rectangular display area of ​​spatially suspended image 3. Consequently, it is difficult for the user to recognize whether character 1532 is deformed due to wide-angle distortion. In this display state, the user is likely to mistakenly believe that character 1532 is larger than character 1531. The following description will describe an example of better display control.

[0335] First, an example of a method for generating a rendered image by shooting a 3D model of a character in a virtual 3D space with a virtual camera is described. FIG. 3Explanations will be given. In the case of generating a rendering image including an object on a virtual 3D space, the range of the image is determined by the angle of view of the virtual 3D camera. The angle of view of the virtual 3D camera can mimic an optical camera in a real space, and is set according to the size of a virtual camera sensor and the focal length of a virtual lens.

[0336] FIG. 3 An example of setting the angle of view of the virtual 3D space camera 1611 and the virtual 3D space camera 1612 with a prescribed camera sensor in a virtual 3D space is shown in FIG. 16. In the example of this figure, the sizes of the camera sensors of the virtual 3D space camera 1611 and the virtual 3D space camera 1612 are the same. In either virtual 3D space camera, the range including the character position 1620 at which a character is disposed is captured as a captured range of the image with the picture width of the picture width 1630.

[0337] Here, the focal length of the virtual 3D space camera 1611 is shorter than the focal length of the virtual 3D space camera 1612. That is, even if the picture width 1630 of the captured range of the image is the same, the angle of view of the virtual 3D space camera 1611 is wider than the angle of view of the virtual 3D space camera 1612. In addition, the relationship between the specific set value of the focal length of the virtual 3D space camera and the angle of view is also affected by the size of the camera sensor. Therefore, the explanation can be simplified by converting the focal length to the case where the camera sensor size is 35 mm film size. Thus, in the following explanation, the focal length after conversion to 35 mm film is used in the explanation.

[0338] Here, the angle of view of the virtual 3D space camera 1611 is wider than the angle of view of the virtual 3D space camera 1612, so there is a possibility that the captured character is distorted by wide-angle distortion at the angle of view of the focal length A of the virtual 3D space camera 1611. In contrast, the angle of view of the virtual 3D space camera 1612 is telephoto (long focus) compared to the angle of view of the virtual 3D space camera 1611, so the wide-angle distortion of the captured character is relatively reduced. In addition, for the background in the virtual 3D space that is farther away from the virtual 3D space camera than the character position 1620, the captured range that can be captured also changes accordingly with the angle of view. However, for a black background space or the like, the user is not likely to recognize the difference in the range of the captured background based on the image of the rendering result.

[0339] As explained above, although it is possible to generate a rendering image using a virtual camera to capture a 3D model of a character on a virtual 3D space, the character in the rendering image is affected by wide-angle distortion depending on the angle of view of the virtual 3D space camera.

[0340] Next, the use of a virtual camera to capture a 3D model of a character on a virtual 3D space will be explained. FIG. 16CAn unnatural occurrence caused by wide-angle distortion and a method for eliminating the same are described in a case where a user views a display screen of a display device such as a spatially-suspended image display device 1000. FIG. 4A-4O An example of an optical image of a display screen of a spatially-suspended image display device viewed by a user, i.e., a spatially-suspended image, is shown. The display screen, i.e., the spatially-suspended image, is rectangular. Let the length of a diagonal line of the rectangle be Pa. In the spatially-suspended image, an image, which is a rendered image obtained by rendering using a perspective projection method, of an object such as a 3D model of a character on a virtual 3D space, photographed by a virtual camera, can be displayed as shown. FIG. 4A-4O

[0341] Here, FIG. 4A-4O In the example, by overlapping a screen of a size Im in a virtual space at the time of generating a rendered image with a display screen of a display device such as the spatially-suspended image display device 1000 in a real space, the field of view angle of a virtual 3D space camera on a virtual 3D space and the field of view angle when a user's eyes in the real space view the display screen are compared. FIG. 16B In the example, as the virtual 3D space camera, three virtual 3D space cameras having different field of view angles, i.e., a virtual 3D space camera 1651, a virtual 3D space camera 1652, and a virtual 3D space camera 1653, are shown.

[0342] In the comparison of the field of view angles shown in FIG. 16D In the comparison of the field of view angles shown in

[0343] In the comparison of the field of view angles shown in In the comparison of the field of view angles shown in

[0344] At this time, in a case where the user views the image from the eye position 1660, wide-angle distortion as in an image rendered by perspective projection using the virtual 3D space camera 1651 does not occur, so the user does not feel the unnaturalness described above. In addition, when the field of view angle is made a telephoto view angle as in the virtual 3D space camera 1653, even if rendering is performed by perspective projection, it approaches parallel projection, but the human eye feels less unnaturalness for parallel projection than for wide-angle distortion of perspective projection.

[0345] Thus, in a case where the user views the image from the eye position 1660, wide-angle distortion as in an image rendered by perspective projection using the virtual 3D space camera 1651 does not occur, so the user does not feel the unnaturalness described above. In addition, when the field of view angle is made a telephoto view angle as in the virtual 3D space camera 1653, even if rendering is performed by perspective projection, it approaches parallel projection, but the human eye feels less unnaturalness for parallel projection than for wide-angle distortion of perspective projection. FIG. 16E

[0346] That is, it can be configured to render an image using a virtual 3D space camera that satisfies, in a case where a virtual 3D space camera whose focal length converted to a 35 mm film is Lf0 has the same field of view angle as when the user views the display screen of the display device such as the spatially-suspended image display device 1000 from the eye position 1660 in the real space, the focal length converted to a 35 mm film Lf of the virtual 3D space camera is Lf ≥ Lf0.

[0347] The focal length Lf0 converted to a 35 mm film of the virtual 3D space camera having the same field of view angle as when the user views the display screen of the display device such as the spatially-suspended image display device 1000 from the eye position 1660 in the real space can be obtained as follows. The field of view angle when the user views the display screen of the display device such as the spatially-suspended image display device 1000 from the eye position 1660 in the real space is equal to the field of view angle of the virtual 3D space camera whose focal length converted to a 35 mm film is Lf0, so the ratio of the visual distance Lm to the diagonal length Pa of the rectangle of the display screen, that is, the spatially-suspended image, is equal to the ratio of the focal length Lf0 converted to a 35 mm film of the virtual 3D space camera to the diagonal length of a 35 mm film.

[0348] ​That is, assuming that the diagonal length of the 35mm film is Fi, Lm / Pa=Lf0 / Fi holds. Transforming it with respect to Lf0, we obtain Lf0=Lm×Fi / Pa. In order to make the field of view Lf of the virtual 3D space camera used in rendering equal to or more telescopic than the field of view when visually viewing the display screen of the spatial suspended image display device 1000 or other display device from the user's eye position 1660 in the real space, it is sufficient to make Lf≥Lf0. Therefore, it is preferable to use a virtual 3D space camera with a focal length that satisfies Lf≥Lm×Fi / Pa converted to 35mm film to shoot objects such as 3D models of characters in the virtual 3D space, render them by perspective projection, and display them on the display screen of the spatial suspended image display device 1000 or other display device.

[0349] By performing the display processing described above, it is possible to display a rendered image such as a 3D model of a character that is less likely to feel unnatural to the user.

[0350] In addition, regarding the visual distance Lm, a technically appropriate distance can be predetermined in the spatial floating image display device 1000. The value of the visual distance Lm can be recorded in FIG. 16D The non-volatile memory 1108 or the storage unit 1170 can be stored. A user manual can be displayed on the display screen of the space floating image 3, etc. to prompt the user of the visual distance Lm and remind the user to use the display device at this distance. FIG. 16B The camera unit 1180 shown in FIG. 1180 performs distance measurement processing, and measures the distance from the display screen such as the space floating image 3 to the user when the user starts using the device, and calculates it as the visual distance Lm. The distance measurement processing can be performed by FIG. 16E The control unit 1110 uses the image captured by the camera unit 1180 to perform.

[0351] Then, in FIG. 16B In the example of visual distance when the display device has an operation detection unit capable of detecting an operation such as a user's finger touching on the display screen, etc. is described. FIG. 16F As shown, it is possible to detect the finger of the user 230 on the display screen ( FIG. 16D The touch display device ( FIG. 16B In the space floating image display device 1000 having an air operation detection sensor 1351 and an air operation detection unit 1350, if it is a touch sensor type, then FIG. 16F The visual distance Lm described in the is determined by the arm's length, including the fingertips. For example, the following URL, as Reference Information 1, contains data collected from approximately 7,000 people during the "Human Body Characteristics Foundation Development Project (size-JPN)" conducted by the Ministry of Economy, Trade and Industry of Japan from 2004 to 2006.

[0352] [Reference 1]

[0353] https: / / warp.ndl.go.jp / info:ndljp / pid / 286890 / www.meti.go.jp / press / 20071001007 / 20071001007.html

[0354] In FIG. 16B , the results of calculating the arm length including the finger length by gender and age group based on the survey results are shown.

[0355] According to the survey results, the group with the longest arm length among the groups classified by gender and age group is the male group of 20 to 24 years old, which is 571 mm on average. The group with the shortest arm length among the groups classified by gender and age group is the female group of 75 to 79 years old, which is 456 mm on average.

[0356] However, when operating a screen by touch operation, the arm is hardly stretched completely to operate. In reality, the device is used in a state where the arm is slightly bent at a distance slightly close to the actual arm length. The distance was actually measured for many people, and the ratio to the arm length was about 80% on average. From these data, it can be estimated that, among the groups classified by gender and age group, the male group of 20 to 24 years old, which has the longest arm length, views the image at a distance Lm of about 457 mm when operating the touch sensor surface, i.e., the image. In addition, it can be estimated that, among the groups classified by gender and age group, the female group of 75 to 79 years old, which has the shortest arm length, views the image at a distance Lm of about 365 mm.

[0357] FIG. 16B-16F is the case where the visual distance Lm of the user of the group with the shortest arm length in the calculation results of FIG. 17A is 365 mm, the table obtained by calculating the specific values of Pa and Lf0 described in FIG. 17A , where Pa is the diagonal length of the rectangle of the display screen, i.e., the space floating image, and Lf0 is the focal length of a virtual 3D space camera having the same field of view angle as the field of view angle when viewing the display screen of the display device 1000 or the like from the user's eye position 1660 in the real space. In FIG. 17A , examples in which the diagonal length Pa of the display screen, i.e., the space floating image, is 10 inches, 5 inches, and 3 inches are shown.

[0358] Here, the table in FIG. 17ACalculation is performed using Lf0 = Lm × Fi / Pa as described in [1]. When the diagonal length Pa of the display screen, i.e., the rectangle of the spatially suspended image, is 10 inches, 5 inches, or 3 inches, the focal length Lf0 is calculated to be 62 mm, 124 mm, or 207 mm in each case. When the spatially suspended image display device 1000 uses the dimensions of each display screen, when generating the image to be displayed on the display screen, a virtual 3D space camera having a focal length Lf converted to 35mm film that is greater than the focal length Lf0 can be used to capture objects such as 3D models of characters in the virtual 3D space, and rendering can be performed using perspective projection. This allows for the display of rendered images such as 3D models of characters that are less likely to appear unnatural to the user.

[0359] Here, substituting the visual distance Lm=365mm and the diagonal length of 35mm film Fi=43.3mm into Lf0=Lm×Fi / Pa, we can derive Lf0=15805 / Pa.

[0360] Therefore, it is sufficient to determine the focal length Lf (mm) of the virtual 3D space camera in image rendering in the manner of Lf ≥ Lf0. Therefore, it can be considered that no matter which display screen, by using the focal length of the virtual 3D space camera of Lf ≥ 15805 / Pa, at least for a specific user group, such a technical effect can be obtained, that is, it is possible to display rendered images such as 3D models of characters that are less likely to feel unnatural when viewed by users.

[0361] then, FIG. 17A It is assumed FIG. 17A In the calculation results, if the visual distance Lm of the user with the longest arm length is 457mm, calculate FIG. 17A The table is obtained by using the specific values ​​of Pa and Lf0 described in , where Pa is the diagonal length of the rectangle of the display screen, i.e., the spatial suspended image, and Lf0 is the focal length of the virtual 3D space camera converted to 35mm film, which has the same field of view as the field of view when visually viewing the display screen of the spatial suspended image display device 1000 or other display device from the user's eye position 1660 in real space. FIG. 15C , examples are shown in which the diagonal length Pa of the display screen, ie, the spatial floating image, is 10 inches, 5 inches, and 3 inches.

[0362] Here, use FIG. 15CLf0 = Lm x Fi / Pa is calculated. In the case where the diagonal length Pa of the rectangular display screen, i.e., the space-suspended image, is 10 inches, 5 inches, and 3 inches, the focal length Lf0 is calculated to be 78 mm, 156 mm, and 259 mm, respectively. In the case where the space-suspended image display device 1000 adopts the size of each display screen, when generating an image to be displayed on the display screen, an object such as a 3D model of a character on a virtual 3D space can be captured by a virtual 3D space camera having a focal length Lf converted to a 35 mm film of 35 mm or more than each focal length Lf0, and rendering is performed by a perspective projection method. In this way, it is possible to display a rendered image of an object such as a 3D model of a character that is less unnatural when viewed by a user.

[0363] Here, Lf0 = Lm x Fi / Pa is calculated. In the case where the diagonal length Pa of the rectangular display screen, i.e., the space-suspended image, is 10 inches, 5 inches, and 3 inches, the focal length Lf0 is calculated to be 78 mm, 156 mm, and 259 mm, respectively. In the case where the space-suspended image display device 1000 adopts the size of each display screen, when generating an image to be displayed on the display screen, an object such as a 3D model of a character on a virtual 3D space can be captured by a virtual 3D space camera having a focal length Lf converted to a 35 mm film of 35 mm or more than each focal length Lf0, and rendering is performed by a perspective projection method. In this way, it is possible to display a rendered image of an object such as a 3D model of a character that is less unnatural when viewed by a user.

[0364] Therefore, as long as the focal length Lf (mm) of the virtual 3D space camera in the image rendering is determined in such a way that Lf ≥ Lf0, it can be considered that, regardless of which display screen, by using a focal length of the virtual 3D space camera of Lf ≥ 19788 / Pa, it is possible to obtain such a technical effect that a rendered image of an object such as a 3D model of a character that is less unnatural when viewed by a user can be displayed for almost all user groups.

[0365] For the above use FIG. 17B The effect of the display method of the rendered image of the character 1531 and the 3D model of the character 1532 in the above-described embodiment will be described using FIG. 17A . FIG. 17B is an example in which rendering is performed by a perspective projection method in a virtual 3D space in which the 3D model of the character 1531 and the 3D model of the character 1532 are present, and the rendered image is displayed on the display device 1.

[0366] Here, FIG. 17B is an example of an image seen by a user in the case where the focal length of the virtual 3D space camera as a rendering viewpoint is set to Lf ≥ Lm x Fi / Pa for rendering processing and display. In this way, even if the perspective projection method is used, as long as the field of view angle of the virtual 3D space camera as a rendering viewpoint is equal to or more telephoto than the field of view angle in which a user does not feel unnatural, the user does not feel unnatural wide-angle distortion.

[0367] In addition, FIG. 17B In the example of

[0368] In addition, in the case where the space-suspended image displayFIG. 18 In the case of an image, the black area of ​​the space floating image 3 is transparent to the user and cannot be seen, and the user cannot recognize the rectangular shape of the outer periphery of the display area of ​​the space floating image 3. FIG. 15A-15D In the example, the user can only see the characters 1531 and 1532 displayed in the space floating image 3 floating in the air. FIG. 17A Images and FIG. 17B The image is different, so the character 1532 will not be displayed larger than the character 1531, so it will not be displayed in FIG. 18 User cognitive problems such as those identified in the images.

[0369] For ease of explanation, FIG. 18 Shown in FIG. 15A-17B An example of displaying a black area within the rectangular display area of ​​the spatial floating image 3 in the image is eliminated.

[0370] Watch FIG. 19A As can be seen from the example of FIG. 19A The black area in the rectangular display area of ​​the spatial floating image 3 is eliminated from the image, and the user will not be able to easily recognize the center of the rectangular display area of ​​the spatial floating image 3. As a result, it is difficult for the user to recognize that the character 1531 is standing in the center of the rectangular display area of ​​the spatial floating image 3 and the character 1532 is standing at the right end of the rectangular display area of ​​the spatial floating image 3. However, in FIG. 3 In the image, since the focal length of the virtual 3D space camera serving as the rendering viewpoint is set to Lf≥Lm×Fi / Pa for rendering processing and display, there will be no wide-angle distortion that the user feels unnatural, and the characters 1531 and 1532 can be better displayed.

[0371] Even if the user does not recognize the rectangular shape of the display area of ​​the spatial floating image 3, they can still better identify characters 1531 and 1532. By performing such display processing, the user can more naturally identify the characters displayed in the spatial floating image 3, further enhancing the real presence of the characters displayed in the air and achieving a better display.

[0372] In addition, FIG. 19A The following shows the description of the specified area in the rectangular display area of ​​the space floating image 3. The specified area refers to such an area that, if displayed in black, will make it difficult for the user to recognize the rectangular shape of the display area of ​​the space floating image 3. FIG. 19B and FIG. 19B 、 FIG. 19AIn the above, an example in which the image in which the area other than the character is entirely displayed in black is explained. However, in a case where a part of the area other than the character is displayed in other than black, there is a case where the user cannot recognize the rectangular shape of the display area of the spatially-suspended image 3.

[0373] In FIG. 19B An example of the area is shown in the middle. FIG. 19C The character 1532 is displayed in the spatially-suspended image 3. The white area 1800 in the background of the character 1532 displays a background image having luminance. In contrast, the upper left vertex 1801 and the upper right vertex 1802 of the rectangle of the display area of the spatially-suspended image 3 are displayed in black.

[0374] In addition, the area 1803 including the edge of the upper side of the rectangle of the display area of the spatially-suspended image 3 is also displayed in black. In this way, in a case where the vertices of the rectangle of the display area of the spatially-suspended image 3 are displayed in black so as to become transparent and invisible from the user, and in a case where the area including one edge of the rectangle of the display area of the spatially-suspended image 3 is displayed in black so as to become transparent and invisible from the user, the elements of the rectangle of the display area of the spatially-suspended image 3 become transparent and invisible from the user. At this time, the user cannot recognize the rectangular shape of the display area of the spatially-suspended image 3.

[0375] As described above, in a case where a certain area (the area satisfies that, if displayed in black, the user cannot recognize the rectangular shape of the display area of the spatially-suspended image 3) is displayed in black, even if it is not an image in which the area other than the character is entirely displayed in black, the effect of the better display processing brought about by using the spatially-suspended image display device 1000 of the present embodiment can be obtained. FIG. 19B The effect of the better display processing brought about by using the spatially-suspended image display device 1000 of the present embodiment is explained.

[0376] Embodiment 4

[0377] Next, as Embodiment 4 of the present application, an example in which the spatially-suspended image display device 1000 explained in each of the drawings of Embodiments 1 to 3 is connected to the Internet, and a new action is performed by connecting to a server mounted with a large-scale language model artificial intelligence via the Internet is explained. As the spatially-suspended image display device 1000 of Embodiment 4 of the present application, any one of the spatially-suspended image display devices 1000 of each of the drawings of Embodiments 1 to 3 can be used. In the present embodiment, the different points from Embodiments 1 to 3 are explained, and the repeated explanation of the same structure as these embodiments is omitted.

[0378] The spatially-suspended image display device 1000 of Embodiment 4 of the present application is connected to the Internet, and a new action is performed by connecting to a server mounted with a large-scale language model artificial intelligence via the Internet. FIG. 19D, an example of the connection between the spatially suspended image display device 1000 and the large-scale language model server 19001 according to Example 4 of the present invention will be described. The spatially suspended image display device 1000 according to Example 4 may also be referred to as a character conversation device. Furthermore, a system including the spatially suspended image display device 1000 and the large-scale language model server 19001 according to Example 4 may be referred to as a character conversation system. The spatially suspended image 3 displayed by the spatially suspended image display device 1000 shows an image of a character 19051. The image of the character 19051 is generated by rendering a 3D model of the character in the virtual space.

[0379] FIG. 19D In the example, the spatial floating image display device 1000 includes a sound output unit 1140 composed of a speaker. In addition, the spatial floating image display device 1000 includes a microphone 1139 that can collect the user's voice. The communication unit 19010 is FIG. 19D The spatial floating image display device 1000 can communicate with a communication device 19011 connected to the Internet 19000 via the communication unit 19010 . FIG. 19D In the example shown, the communication between the communication unit 19010 and the communication device 19011 is shown as an example of wireless communication, but wired communication is also possible. The communication path from the communication unit 19010 to the Internet 19000 can include wired and wireless portions. The spatial floating image display device 1000 can communicate with the large-scale language model server 19001 via the communication device 19011 and the Internet 19000. In addition, the spatial floating image display device 1000 can communicate with a second server 19002 different from the large-scale language model server 19001 via the communication device 19011 and the Internet 19000. The structure including the spatial floating image display device 1000 and the large-scale language model server 19001 can be regarded as a single system.

[0380] Next, use FIG. 19D , an example of the operation of the character conversation device (space floating image display device 1000) of Example 4 of the present invention is described. This can be considered as an example of the operation of the character conversation system including the space floating image display device 1000 and the large-scale language model server 19001. In addition, FIG. 19E Omitted FIG. 19E 19000 and other communication paths shown in FIG. FIG. 19E , a user 230 of the spatial floating image display device 1000 is also shown.

[0381] First, the large-scale language model server 19001 is a server equipped with a large-scale language model artificial intelligence. The large-scale language model is also referred to as LLM (Large Language Model). Specifically, various models such as GPT-1, GPT-2, GPT-3, InstructGPT, ChatGPT, and the like are disclosed. These technologies can be used in the present embodiment. These large-scale language models are artificial intelligence models generated by large-scale pre-training on a large amount of documents, natural language contained in texts, and the like present in human society. The number of parameters of the artificial intelligence model is over one hundred million. Further, in addition to this, there are models in which reinforcement learning based on human feedforward is implemented. An example of the underlying model is a model called Transformer and the like. As an example of training of these models, for example, reference document 1 and the like are disclosed.

[0382] [Reference Document 1]

[0383] Long Ouyang, et. al. “Training language models to follow instructions with human feedback”, https: / / arxiv.org / abs / 2203.02155

[0384] These large-scale language models can perform translation of natural language as an object, article correction of natural language as an object, article summarization of natural language as an object, and the like. Among them, a more advanced model can also answer a question in natural language (also referred to as a dialogue or a conversation), generate a suggestion in natural language, generate a programming code, and the like. Because the number of parameters of these artificial intelligence models is very large, training requires a huge amount of data and computing resources. Therefore, if the artificial intelligence training of this level is performed for a specific purpose, the resource efficiency is very poor. For this reason, it is more efficient in terms of resources to generate a model by performing large-scale training as a Foundation Model that can be applied to various purposes, and to use it through various terminals via an API (Application Programming Interface).

[0385] Here, a series of flows of the operation of the spatial floating image display device 1000 will be described. In addition, the spatial floating image display device 1000 loads a character action program stored in the storage section 1170 or the like into the memory 1109, and the control section 110 executes the character action program, whereby various processes described below can be implemented.

[0386] First, the spatial floating image display device 1000 has a microphone 1139, and when the user 230 speaks to the character 19051, the user's voice (the user's utterance) is collected by the microphone 1139 and converted into a voice signal. Here, the character action program executed by the control section 110 extracts the text of the utterance spoken by the user 230 from the voice signal. The text is natural language. In addition, the text extraction of the utterance spoken by the user 230 can be continuously performed on the entire utterance, or can start the extraction when the user speaks in a predetermined period after a keyword as a trigger. For example, the keyword as a trigger can be a case where the user speaks the character's name or the like after "Hello". For example, assuming that the name of the character 19051 is "Koto", "Hello, Koto!" can be used as a keyword as a trigger.

[0387] The character action program of the spatial floating image display device 1000 generates a prompt based on the text of the utterance spoken by the user 230, and transmits the prompt to the large-scale language model server 19001 using an API. Here, the prompt can be metadata or the like in which information is saved in a tag of a markup language or a JSON or the like. In the prompt, natural language text information is saved as a main message. As the types of the prompt transmitted from the spatial floating image display device 1000 to the large-scale language model server 19001, there are a setting prompt in which an instruction such as an initial setting is saved, and a user prompt which reflects the user's instruction. It is also possible to save type identification information which identifies whether the prompt is a setting prompt or a user prompt, in a part other than the main message of the prompt. When the character action program of the spatial floating image display device 1000 generates a prompt based on the text of the utterance spoken by the user 230, a user prompt is generated and transmitted to the large-scale language model server 19001.

[0388] Next, the artificial intelligence large-scale language model of the large-scale language model server 19001 performs reasoning based on the prompt transmitted from the spatial floating image display device 1000, and generates an answer including natural language text information based on the result thereof. The large-scale language model server 19001 transmits the answer to the spatial floating image display device 1000 using an API. In the answer, natural language text information is saved as a main message. Here, the answer can be metadata in which information is saved in a tag of a markup language or a JSON or the like in the same format as the above-described prompt. In the answer, in the case of using the same format as the above-described prompt, in order to indicate that it is information of a different type from the above-described initial setting prompt and user prompt, type identification information can be saved in a part other than the main message. For example, information indicating that it is a response sentence from the large-scale language model is saved.

[0389] Next, the spatially-suspended image display device 1000 receives the answer from the large-scale language model server 19001, and extracts the natural language text information held as the main message in the answer. The character action program of the spatially-suspended image display device 1000 generates a natural language sound (voice) as an answer to the user based on the natural language text information extracted from the above answer using a sound synthesis (voice synthesis) technique in a manner that sounds as if it is the voice of the character 19051, and outputs it from the speaker, i.e., the sound output unit 1140. This process can be described as the character "speaking".

[0390] In FIG. 19D Conversations Examples 1 to 5, a specific example of the answer sound of the character 19051 corresponding to the utterance of the user 230 obtained by the process of the spatially-suspended image display device 1000 and the large-scale language model server 19001 described above is shown. In this way, the user 230 can have a conversation in a manner that the character 19051 seems to be an actually existing person.

[0391] According to the spatially-suspended image display device 1000 or the system including the spatially-suspended image display device 1000 described above, FIG. 19E does not need to mount the large-scale language model itself that needs a large amount of data and a large amount of computing resources for training in the spatially-suspended image display device 1000 itself. Also, the ability of the advanced natural language processing using the large-scale language model can be used via the API, and in the case where the user speaks to the character, a better answer to the user and a better conversation can be performed.

[0392] Next, an example of the action of the character conversation device (spatially-suspended image display device 1000) of Embodiment 4 of the present application will be described using FIG. 19E It can be considered that this is a description of an example of the action of the character conversation system including the spatially-suspended image display device 1000 and the large-scale language model server 19001. Specifically, FIG. 19D is an example of the natural language text of the main message of the prompt word sent from the spatially-suspended image display device 1000 to the large-scale language model server 19001 and the natural language text of the main message of the server response as an answer thereto, and the conversation between the character 19051 displayed on the spatially-suspended image display device 1000 and the user 230 is implemented based on this.

[0393] In addition, FIG. 19D In the above, the case where the interaction of the prompt word and the answer is performed in time series from the displayed setting prompt word, the first user prompt word, and the answer thereto, to the fourth user prompt word and the answer thereto is shown.

[0394] As FIG. 19EAs the initial setting, the large-scale language model server 19001 can be instructed by the setting prompt word to indicate the name of the large-scale language model itself, the role to be played, the characteristics of the conversation, and the like. In addition, it can also be made to understand the name of the user as the initial setting. Thus, the large-scale language model generates the first and subsequent answers while maintaining the role. In this way, when the user hears the voice of the role 19051 based on the first and subsequent answers, it can feel as if the role 19051 is the setting and personality of the character described in the setting prompt word. In addition, the large-scale language model server 19001 of the present embodiment is configured to store the content of the conversation until the end of the series of conversations in the memory, and generate an answer based on the stored series of user prompt words and their answers. Thus, it is possible to realize a conversation as shown in FIG. 19D

[0395] Next, an example of the action of the role conversation device (the spatial floating image display device 1000) of Embodiment 4 of the present application will be described using FIG. 19D It can be considered as an example of the description of the action of the role conversation system including the spatial floating image display device 1000 and the large-scale language model server 19001. Specifically, FIG. 19E is an example of the natural language text of the main message of the prompt word transmitted from the spatial floating image display device 1000 to the large-scale language model server 19001 and the natural language text of the main message of the server response as the answer, and the conversation between the role 19051 displayed on the spatial floating image display device 1000 and the user 230 is realized based on this.

[0396] FIG. 19F represents an example in the case where the user 230 newly starts a conversation again after the series of conversations shown in FIG. 19F FIG. 19F

[0397] ​​​Here, the "end" of the "series of conversation continuations" refers to a process in which the large-scale language model server 19001 clears the memory of the conversation held while the series of conversation continuations is continued from the large-scale language model server 19001, in a case where a prescribed condition is satisfied. As an example of the prescribed condition, for example, there can be mentioned a case where the large-scale language model server 19001 is instructed by the spatial floating image display device 1000 through a prompt word that the "end" of the "series of conversation continuations" has been made. In addition, as another example of the prescribed condition, for example, there can be mentioned a case where a prescribed time or more elapses after the prompt word is no longer transmitted from the spatial floating image display device 1000 to the large-scale language model server 19001 with respect to the series of conversation continuations (timeout). In addition, there can also be mentioned a case where the authentication process in the connection between the spatial floating image display device 1000 and the large-scale language model server 19001 is invalidated due to a communication interruption, a power-off (OFF) of the spatial floating image display device 1000, or the like, when the above-described interaction of the prompt word and the answer is made on the basis of the authentication process.

[0398] In addition, when the "series of conversation continuations" "ends", the large-scale language model server 19001 clears the memory of the conversation held while the series of conversation continuations is continued from the large-scale language model server 19001. Thereby, FIG. 19D The conversation illustrated in FIG. 19E is made after the series of conversation illustrated in FIG. 19F is a response to the content in a state where the large-scale language model itself, the character to be played, the conversation characteristics, the name of the user, and the like are completely forgotten from the set prompt word. Also, FIG. 19D The conversation illustrated in FIG. 19D is a response to the content in a state where the series of conversation illustrated in FIG. 19F is completely forgotten. That is, because FIG. 19F the "series of conversation continuations" has "ended",

[0399] This results in a feeling for the user 230 that the character 19051 has lost the memory about himself or has become another person. From the user 230, the response to the character will be strongly unnatural, and a feeling of loneliness and disappointment will be experienced. Under such an action, there is a problem that the name, the identity, or the conversation characteristics, the personality, and the like of the character 19051 displayed on the spatial floating image display device 1000 cannot be ensured to be consistent with the setting and the memory.

[0400] Next, the use of FIG. 19FAn example of the action of the role conversation device (spatially suspended image display device 1000) of Embodiment 4 of the present application will be described. It can be considered that this is a description of an example of the action of a role conversation system including the spatially suspended image display device 1000 and the large-scale language model server 19001. Specifically, FIG. 19G is an example of the natural language text of the prompt word main message transmitted from the spatially suspended image display device 1000 to the large-scale language model server 19001 and the natural language text of the server response main message as an answer thereto, and the conversation between the role 19051 displayed on the spatially suspended image display device 1000 and the user 230 is implemented based on this.

[0401] FIG. 19G indicates an example of a case where the user 230 newly starts a conversation with the role 19051 after the series of conversations shown in FIG. 19G and after the continuation of the series of conversations ends. Unlike the processing of FIG. 19F , in the processing of FIG. 19G , at the time of newly starting a conversation, the spatially suspended image display device 1000 transmits a set prompt word as an initial prompt word to the large-scale language model server 19001. In this set prompt word, the same natural language text as the set prompt word of the initial setting of FIG. 19F is saved. This can be expressed as a re-set text. In this set prompt word, a natural language text explaining the past conversation history is then saved. This can be expressed as a conversation history text. As for the past conversation history, it can be recorded in the storage section 1170 by the spatially suspended image display device 1000 as natural language text information associated with the information of the date and time of the conversation during the period of the series of conversations of the conversation explained in FIG. 19F . In the case where there are conversations of different dates, the history of the conversation can be recorded in association with the information of the date and time for each conversation, respectively, and accumulated. In generating the set prompt word of the initial prompt word of the later conversation as shown in FIG. 19F , the natural language text information of the conversation and the information of the date and time of the conversation recorded in the storage section 1170 can be read for generating this set prompt word.

[0402] In addition, in the case where the natural language text information of the past conversation history is to be used in generating this set prompt word, since it is data transmitted to the large-scale language model, the format can be freely determined to some extent, like FIG. 19HAs shown, the natural language conjunction or postposition such as "I said this on ○ month ○ day," "You said this on ○ month ○ day," and the like can be prepared, and the processing of the text of the setting prompt word is performed in fusion with the natural language text information of the recorded conversation. In addition, the date and time information of the conversation read from the storage section 1170 can be used as a part of the text of the setting prompt word in fusion with the above-mentioned "○ month ○ day" portion and the like.

[0403] Even in the case where the user 230 newly performs a conversation by speaking to the character 19051 again after a series of conversations and the continuation of the series of conversations ends, by performing the generation processing and the transmission processing of the setting prompt word described above, the answer of the user prompt word thereafter will be able to reflect the identity, name, conversation characteristics, personality, and / or conversation characteristics and the like of the character at the time of the previous conversation, the setting, and the conversation history. Thereby, from the user, the consistency of the identity, name, conversation characteristics, or personality and the like of the character at the time of the previous conversation, the setting, and the memory can be recognized to be better ensured, so it is more preferable. FIG. 19H

[0404] Next, an example of the action of the character conversation device (the spatial floating image display device 1000) of Embodiment 4 of the present application will be described using FIG. 19H It can be considered that this is a description of an example of the action of the character conversation system including the spatial floating image display device 1000 and the large-scale language model server 19001. Specifically, ​ is an example of the natural language text of the main message of the prompt word transmitted from the spatial floating image display device 1000 to the large-scale language model server 19001 and the natural language text of the main message of the server response as the answer thereof, and the conversation between the character 19051 displayed on the spatial floating image display device 1000 and the user 230 is realized based on this.

[0405] ​ Indicates ​ As shown in the series of conversations, an example of a series of conversations from the first user prompt word and its answer to the third user prompt word and its answer after the initial setting prompt word. ​ In the series of conversations, the case where the interaction of the prompt word and the answer is performed in time series is shown. The content of the setting prompt word is the same as shown in ​ is omitted.

[0406] As shown in the natural language text of the server response of the table of ​ By using ​ ​The server response of the large-scale language model AI of large-scale language model server 19001 will be able to reflect the settings and conversation history of the character's identity, name, conversation characteristics, or personality from previous conversations. This allows the user to better ensure the consistency of the settings and memories of the character's identity, name, conversation characteristics, or personality from previous conversations, making it more preferred. Furthermore, the user can perceive the character as being the same person, which can also be referred to as class consistency of the character from the user's perspective.

[0407] In addition, from the user's perspective, being able to share memories with the character can provide a more enjoyable character conversation experience.

[0408] Next, use ​ , an example of the operation of the character conversation device (space floating image display device 1000) of Example 4 of the present invention is described. This can be considered as an example of the operation of the character conversation system including the space floating image display device 1000 and the large-scale language model server 19001. Specifically, ​ This figure shows an example of an operation for switching characters to be displayed in the spatial floating image 3 of the spatial floating image display device 1000 from among a plurality of character candidates. The character operation program executed by the control unit 110 of the spatial floating image display device 1000 can switch the displayed characters based on, for example, an operation input to the operation input unit 1107 or an operation detected by the mid-air operation detection unit 1350.

[0409] ​ In the example, except Figures 19A-19G In addition to the character 19051 (named "Koto") used in the description, the character 19052 (named "Tom") and the character 19053 (named "Necco") are also shown. The characters 19051 (named "Koto") and 19052 (named "Tom") are human characters, and the character 19053 (named "Necco") is a cat character. Regarding the display switching of the characters displayed in the spatial floating image 3, the image generated by rendering the characters in the different virtual 3D spaces for each character can be switched on the display device 1. Regarding the processing of the rendered image for displaying the 3D model of each character, for example, Figure 15A In addition, depending on the character, a 2D image that moves dynamically can also be displayed.

[0410] In addition, the character action program executed by the control section 110 preferably changes the synthesized sound (synthesized speech) used in the "utterance" of each character when the display of the character displayed in the space floating image 3 is switched. In this case, the data of the synthesized sound of the tone associated with each character can be stored in the storage section 1170 in advance, and the synthesized sound change processing is performed when the display of the character is switched.

[0411] In addition, Figure 19H The example of the space floating image display device 1000 is configured so that the user 230 can have a conversation with any one of the characters. Figure 19H The space floating image display device 1000 sets different identities, names, conversation characteristics, or personalities, etc. for each of the characters. In addition, the memory of each character based on the conversation history is also managed as different data for each character.

[0412] For this reason, the space floating image display device 1000 constructs the database 19200 shown in FIG. 17 in the storage section 1170, and manages the settings of the characters and the conversation history of the characters with the database. Figure 19I

[0413] Next, an example of the action of the character conversation device (the space floating image display device 1000) of Embodiment 4 of the present application will be described using FIG. 18. Figure 19I Figure 19I is a diagram of the database 19200 for managing the settings of the characters and the conversation history of the characters for a plurality of characters displayed in the space floating image 3 of the space floating image display device 1000.

[0414] The character action program executed by the control section 110 of the space floating image display device 1000 constructs the database 19200 in the storage section 1170, for example. The character ID is an identification number that respectively identifies a plurality of characters that can be displayed in the space floating image display device 1000, and can be a natural number or can use letters, etc. The name is data of the name of each of a plurality of characters that can be displayed in the space floating image display device 1000.

[0415] The initial setting prompt word is natural language text information that explains the setting of the character identity, name, conversation characteristic, or personality, etc. of each of a plurality of characters that can be displayed in the space floating image display device 1000. This initial setting prompt word is natural language text information of the main data of the setting prompt word transmitted from the space floating image display device 1000 to the large-scale language model server 19001, so it is preferable that the description content can be directly read by the artificial intelligence large-scale language model of the large-scale language model server 19001. ​​

[0416] Conversation histories 1, 2, and so on, record conversations between each character and the user, recorded on a role-by-role basis. These conversation histories are contained in the natural language text information of the main data for setting prompt words, which is sent from the spatially suspended image display device 1000 to the large-scale language model server 19001. Therefore, it is preferable to use content that can be directly read by the artificial intelligence large-scale language model of the large-scale language model server 19001.

[0417] The character action program executed by the control unit 110 of the spatial floating image display device 1000 uses the character action program to switch the character displayed in the spatial floating image 3 of the spatial floating image display device 1000. Figure 19I The database 19200 selects and switches the initial setting prompt words and conversation history used in the natural language text information of the main data of setting prompt words sent from the spatial floating image display device 1000 to the large-scale language model server 19001 in a manner corresponding to the characters displayed in the spatial floating image 3 of the spatial floating image display device 1000. In addition, the character action program records the conversation history in the database 19200 each time the user 230 has a conversation with the character. Figure 19I In the area of ​​the conversation history corresponding to the character displayed in the spatial floating image 3 in the database 19200.

[0418] By using database 19200 in this manner, the character action program executed by control unit 110 of spatially suspended image display device 1000 allows the user 230 to establish a conversation with the character using the character's speech, using the same large-scale artificial intelligence language model based on the same large-scale language model server 19001. However, the user can perceive that the uniqueness of each character's personality settings is maintained, and each character maintains a memory of their own unique conversations. This is more preferable because the user can better ensure consistency between the settings and memories of each character's identity, name, conversational characteristics, or personality from previous conversations. This can be expressed as ensuring class consistency for each character as seen by the user.

[0419] Thus, even if the spatial floating image display device 1000 is configured to be able to switch the character to be displayed in the spatial floating image 3 from a plurality of character candidates, in this case, through the actions using the database 19200 described above, from the user's perspective, the conversation with each character will feel less unnatural, and memories can be shared with multiple characters respectively, resulting in a more pleasant character conversation experience.

[0420] In addition, by making it impossible for the user to edit the initial setting prompt words of the plurality of characters, the settings of the identity, name, conversation characteristics, or personality, and the like of each character can be maintained in a state close to the intention of the provider of the spatial floating image display device 1000 or the content producer of the character. In contrast, it is also possible to enable the user to edit the initial setting prompt words of the character by inputting through the operation input section 1107 or the like. In this case, the settings of the identity, name, conversation characteristics, or personality, and the like of the character can become the settings preferred by the user, and the user can have a conversation with the character set by the user. In this case, it is also possible to replace the 3D model or the rendered image thereof of the character and the type of the synthesized sound of the character accordingly.

[0421] Next, the operation of the character conversation device (the spatial floating image display device 1000) of Embodiment 4 of the present application will be described using Figure 19J

[0422] As described above, for a large-scale language model, the resource efficiency is very poor when the artificial intelligence is trained for a specific purpose. Therefore, it is more efficient in terms of resources to generate a model by training a large-scale language model as a Foundation Model that can be applied to various purposes and use it through various terminals via an API (Application Programming Interface). In this way, the provider of the large-scale language model mostly recovers the cost of training the large-scale language model from the user of the terminal in the form of an API usage fee for the terminal. At this time, the natural language model mostly requires payment of the API usage fee in terms of the number of tokens, which is a language unit obtained by dividing an article. Figure 19B

[0423] For this reason, in the spatial floating image display device 1000 of Embodiment 4 of the present application as well, by reducing the number of tokens in the natural language text information transmitted using the API between the spatial floating image display device 1000 and the large-scale language model server 19001, it is possible to provide the character conversation service implemented by the character conversation device based on the spatial floating image display device 1000 and / or the character conversation system based on the spatial floating image display device 1000 and the large-scale language model server 19001 to the user at a lower cost. ​​

[0424] For example, by employing the processes and structures of Examples 1 to 3 shown in the table as Figure 19J

[0425] Example 1 is an example of reducing the number of word units in the session history text saved and transmitted in the setting prompt words of the API, using document summarization processing to shorten the session history text and reduce the number of word units. For example, the natural language of the session history of the character recorded in the storage section 1170 is summarized and recorded again. The article summarization can be performed at the start of the next session, but it is more time-efficient to perform it at the end of the “series of sessions”.

[0426] In addition, the article summarization processing can also be commissioned to the large-scale language model itself of the large-scale language model server 19001 to perform summarization. However, in this case, the word unit saving effect is low. Therefore, for example, in the case where the article summarization processing of the natural language is provided at a lower cost in the second server 19002 via the API than in the large-scale language model of the large-scale language model server 19001, the article summarization processing can be commissioned to the second server 19002 via the API, and the article abstract of the session history is saved in the setting prompt words of the large-scale language model server 19001 and transmitted.

[0427] In addition, if only the article summarization processing, the terminal-side processing is also possible, and the article summarization can be performed by the document summarization program loaded into the memory 1109 of the spatially suspended image display device 1000 executed by the control section 110. In this case, the word unit saving effect is high. In addition, even if the history of the session becomes long, as long as the upper limit of the number of characters after summarization is specified in the article summarization processing, the upper limit of the article length of the session history can be determined, so the upper limit value of the word units can be specified, and the word units can be saved.

[0428] In addition, the text information of the initial setting of the character, such as the identity, name, conversation characteristics, or personality, does not increase like the session history, so it is efficient and preferable to maintain the description of the text information of the initial setting prompt words of the character and reduce the number of word units of the text information of the session history.

[0429] The processing explained in Example 1 can be performed by the character action program executed by the control section 110 controlling each section.

[0430] ​Example 2 is another example of reducing the number of tokens in the conversation history text stored and transmitted in the setting prompt words of the API. For example, in the conversation history with the character recorded in the storage unit 1170, the older history is cleared to reduce the number of tokens. If the upper limit of the number of characters in the conversation history is specified, the upper limit of the length of the article in the conversation history can be determined, so the upper limit value of the tokens can be specified, and tokens can be saved. Alternatively, a method of specifying a specified period of the conversation history and deleting the conversation history that exceeds the period can be adopted. In this case, tokens can also be saved. In addition, in Example 2, the character's initial setting text information such as the character's identity, name, conversation characteristics, or personality will not increase like the conversation history, so it is efficient and preferred to maintain the description of the text information of the character's initial setting prompt words and reduce the number of tokens in the text information of the conversation history.

[0431] The processing described in Example 2 can be performed by controlling each part through the character motion program executed by the control unit 110 .

[0432] Example 3 is a method for reducing the number of word units by reducing the frequency of sending setup prompts using the API. Specifically, after the device is powered on and the displayed character is switched, even after the image and synthesized voice settings for the displayed character are complete, setup prompts are not sent in advance. Instead, setup prompts are sent to large-scale language model server 19001 only when control unit 110 determines that the natural language text information contained in the user's voice captured by microphone 1139 is text information for use with an artificial intelligence large-scale language model. This reduces the frequency of sending setup prompts to large-scale language model server 19001 and reduces the number of word units.

[0433] Specifically, for example, after the device is powered on (ON) and an operation input is made to switch the display character, the display processing of the display device 1 under the control of the character action program executed by the control unit 110 is performed, such as Figure 19H As shown, character 19051 (named "Koto") is displayed in spatial floating image 3. At this point, if a synthesized voice corresponding to character 19051 is stored and prepared in storage unit 1170 or the like, a synthesized voice corresponding to character 19051's appearance, such as "Good morning. I'm Koto," "Hello. I'm Koto," or "Good evening. I'm Koto," can be output from sound output unit 1140, i.e., the speaker. At this point, the image of character 19051 has already been set as the image of the character displayed in spatial floating image 3, and the synthesized voice corresponding to character 19051 has been set as the synthesized voice output from sound output unit 1140, i.e., the speaker.

[0434] Here, for the inference processing of the artificial intelligence large-scale language model in the large-scale language model server 19001 that has been explained, it is also the case that the longer the prompt word, the more time-consuming it is. In particular, in the case where the text information about the past conversation history is included in the set prompt word, the amount of word units of the prompt word increases, so in particular, the inference processing time will be extended. The set prompt word itself and its answer are not output to the user 230. From the start of the answer of the user prompt word after the set prompt word, the synthesized voice of the "utterance" of the character is output from the sound output unit 1140, that is, the speaker. In this way, if the set prompt word is transmitted from the spatial floating image display device 1000 to the large-scale language model server 19001 in advance, and the inference processing of the large-scale language model for the set prompt word is completed in advance, the output response of the synthesized voice of the "utterance" of the character 19051 is faster after the user 230 speaks to the character 19051, so it seems more optimal at first glance.

[0435] However, in the case where the set prompt word is transmitted to the large-scale language model server 19001 before the user 230 speaks, and the inference processing of the large-scale language model for the set prompt word is completed in advance, there can be a case where, for example, the user 230 turns off (OFF) the power of the spatial floating image display device 1000 by operating the operation input unit 1107 or the air operation detection unit 1350, and a case where, for example, the user 230 switches the display character from the character 19051 to another character by operating the operation input unit 1107 or the air operation detection unit 1350. In this case, the number of word units that are processed in advance by transmitting the set prompt word to the large-scale language model server 19001 and processing by the inference processing of the large-scale language model will be the number of word units that are wasted in terms of usage fees. This will hinder the provision of the character conversation service that is implemented by the character conversation device based on the spatial floating image display device 1000, and / or the character conversation system based on the spatial floating image display device 1000 and the large-scale language model server 19001 to the user at a lower cost.

[0436] Therefore, the spatial floating image display device 1000 is preferably maintained in a state where the set prompt word is not transmitted to the large-scale language model server 19001 until the time when the user 230 speaks to the character 19051 is recognized, even after the image of the character 19051 is set as the image of the character displayed in the spatial floating image 3 and the synthesized voice output from the sound output unit 1140, that is, the speaker, is set to the synthesized voice corresponding to the character 19051 by the control of the character action program executed by the control unit 110 after the power of the device is turned on (ON) and the operation input of the display character switching is performed.

[0437] Here, the time when the user 230 speaks to the character 19051 is recognized, for example, can be the time when the user 230 is detected to speak to the character 19051 by the voice input unit 1120, or the time when the user 230 is detected to speak to the character 19051 by the air operation detection unit 1350. Figure 19Bthe time when the user 230 utters the utterance, and the like. In this way, the number of processing tokens that waste the usage fee can be reduced, and the user can be provided with the character conversation service implemented by the character conversation device based on the spatial floating image display device 1000 and / or the character conversation system based on the spatial floating image display device 1000 and the large-scale language model server 19001 at a lower cost.

[0438] Further, next, even after the time when the user 230 is recognized to speak to the character 19051, in a case where, for example, the text information extracted from the sound of the user 230 collected by the microphone 1139 is text information corresponding to a preset keyword that does not require inference processing of the large-scale language model, it is preferable to maintain a state in which the set prompt word is not transmitted to the large-scale language model server 19001. Specifically, as an example of the preset keyword, a case where the user 230 requests the character 19051 to make a reaction (for example, an animation in which the character 19051 moves or a synthetic sound is emitted, and the like) can be given. In this case, the character action program executed by the control section 110 can read the action data corresponding to the character 19051, the animation image, and / or the synthetic sound data corresponding to the reaction saved in the storage section, and using these data, perform the generation processing of the image displayed in the spatial floating image 3 and the output processing of the synthetic sound output from the sound output section 1140, that is, the speaker.

[0439] Such processing does not necessarily require inference processing of the large-scale language model of the large-scale language model server 19001. In a case where the user 230 turns off (OFF) the power of the spatial floating image display device 1000 by operating the operation input section 1107 or the air operation detection section 1350 after the processing, and / or in a case where, for example, the user 230 switches the display character from the character 19051 to another character by operating the operation input section 1107 or the air operation detection section 1350, if the set prompt word has been transmitted to the large-scale language model server 19001 in advance and processed by inference processing of the large-scale language model, the number of tokens of the processing also becomes the number of processing tokens that waste the usage fee.

[0440] Therefore, it is preferable that the state of not transmitting the set prompt word to the large-scale language model server 19001 is maintained even after the time point at which the user 230 is recognized to speak to the character 19051 described above until, for example, a time point at which it is judged whether or not the text information extracted from the user's 230 voice collected by the microphone 1139 corresponds to the preset keyword for which the inference processing using the large-scale language model is not required. It is preferable that the set prompt word is transmitted to the large-scale language model server 19001 and the inference processing using the large-scale language model is performed only in a case where it is judged by the judgment that the inference processing using the large-scale language model is required.

[0441] In addition, the processing described in Example 3 can be performed by the character action program executed by the control section 110 controlling each section.

[0442] According to the method of reducing (saving) the number of processing tokens of the large-scale language model of each of the above-described Figure 19J According to the method of reducing (saving) the number of processing tokens of the large-scale language model of each of the above-described

[0443] According to the character conversation device and the character conversation system of Embodiment 4 described above, it is possible to further reduce the unnaturalness that the user feels from the conversation with the character displayed on the display device. In addition, according to the character conversation device and the character conversation system of Embodiment 4, it is possible to provide the user with the character conversation service at a lower cost.

[0444] In addition, for the display device of the character conversation device and the character conversation system of Embodiment 4, a spatially floating image display device is described as an example thereof. However, Embodiment 4 does not necessarily use the display device that displays a spatially floating image in the air. For example, a display device that displays an image on a physical surface, such as a liquid crystal panel, an organic EL panel, a plasma display, a projector that reflects an image on a non-transparent screen to display the image, a projector that diffuses an image on a transparent screen to display the image, and the like can be used. In this case, although there are various forms of display screens of the displayed image, the portion of each of these display devices that displays an image that the user can see can be referred to as a display section.

[0445] In this case, according to the character conversation device and the character conversation system of Embodiment 4, it is possible to further reduce the unnaturalness that the user feels from the conversation with the character displayed on the display device. In addition, according to the character conversation device and the character conversation system of Embodiment 4, it is possible to provide the user with the character conversation service at a lower cost.

[0446] In the technology of the present embodiment, by causing high-resolution and high-brightness image information to be displayed in a state of floating in space, for example, a user can operate without feeling uneasy about contact infection of infectious diseases. If the technology of the present embodiment is used in a system used by a large number of users who are uncertain, the risk of contact infection of infectious diseases can be reduced, and a non-contact user interface that can be used without feeling uneasy can be provided. Thus, "3 Good Health and Well-Being" of the Sustainable Development Goals (SDGs) advocated by the United Nations is facilitated.

[0447] In addition, in the technology of the present embodiment, by reducing the divergence angle of the outgoing image light and further unifying it into a specific polarization, only the normally reflected light with respect to the retroreflective sheet is efficiently reflected, so the light utilization efficiency is high, and a bright and clear spatially floating image can be obtained. According to the technology of the present embodiment, a non-contact user interface with excellent usability, in which power consumption can be greatly reduced, can be provided. Thus, "9 Industry, Innovation and Infrastructure" and "11 Sustainable Cities and Communities" of the Sustainable Development Goals (SDGs) advocated by the United Nations are facilitated.

[0448] The above describes various embodiments in detail, but the present application is not limited only to the above-described embodiments, and various modifications are included. For example, the above-described embodiments describe the entire system in detail in order to easily and clearly explain the present application, but are not limited to necessarily having all the structures described. In addition, a part of the structure of one embodiment can be replaced with the structure of another embodiment, and the structure of another embodiment can be added to the structure of one embodiment. In addition, with respect to a part of the structure of each embodiment, other structures can be added, deleted, or replaced.

[0449] Explanation of Reference Signs

[0450] 1 … display device, 2 … retroreflective sheet (retroreflective sheet), 3 … spatial image (spatially floating image), 105 … window glass, 100 … transparent member, 101 … polarization separation member, 101B … polarization separation member, 12 … absorption-type polarizing plate, 13 … light source device, 54 … light direction conversion panel, 151 … retroreflective sheet, 102, 202 … LED substrate, 203 … light guide body, 205, 271 … reflective sheet, 206, 270 … phase difference plate, 230 … user, 1000 … spatially floating image display device, 1110 … control section, 1160 … image control section, 1180 … imaging section, 1102 … image display section, 1350 … air operation detection section, 1351 … air operation detection sensor.

Claims

1. An air-suspended image display device, characterized in that: include: Image Processing Department; a display unit for displaying the image processed by the image processing unit; and An optical system that generates an image suspended in the air based on the image displayed by the display unit, wherein: The image processing unit performs image processing on an image generated by rendering a virtual 3D space in which an object having a character is arranged. The rendering of the virtual 3D space is performed by shooting with a virtual 3D space camera set in the virtual 3D space through perspective projection. The image generated by the rendering and displayed on the display unit is an image obtained by shooting the virtual 3D space in which the object of the character is configured, with the field of view of the virtual 3D space camera set in a manner satisfying Lf≥Lm×Fi / Pa, assuming that the focal length of the lens of the virtual 3D space camera is converted to 35mm film as Lf, the distance at which the user of the aerial floating image display device visually views the display screen of the aerial floating image, i.e., the visual distance, is Lm, the diagonal length of the 35mm film is Fi, and the diagonal length of the display screen of the aerial floating image is Pa.

2. The mid-air floating image display device according to claim 1, wherein: The display screen of the aerial floating image has a rectangular display area. In the background of the character in the image generated by the rendering, at least one vertex of the rectangle of the display screen of the image floating in the air is displayed in black, and is in a state that is invisible to the user in the air.

3. The mid-air floating image display device according to claim 1, wherein: The display screen of the aerial floating image has a rectangular display area. In the background of the character in the image generated by the rendering, at least one side of the rectangle of the display screen of the image floating in the air is included in a black display area and is not visible to the user in the air.

4. An air-suspended image display device, characterized in that: include: Image Processing Department; a display unit for displaying the image processed by the image processing unit; an optical system for generating an image suspended in mid-air based on the image displayed on the display unit; and An operation detection unit detects the operation performed by the user on the display screen of the air-floating image, wherein: The image processing unit performs image processing on an image generated by rendering a virtual 3D space in which an object having a character is arranged. The rendering of the virtual 3D space is performed by shooting with a virtual 3D space camera set in the virtual 3D space through perspective projection. The image generated by the rendering and displayed on the display unit is an image obtained by shooting the virtual 3D space in which the character is arranged, with the field of view of the virtual 3D space camera set so as to satisfy Lf ≥ 15805 / Pa, assuming that the focal length of the lens of the virtual 3D space camera converted to 35mm film is Lf and the diagonal length of the display screen of the suspended image is Pa. The units of Lf and Pa are mm.

5. The mid-air floating image display device according to claim 4, wherein: The display screen of the aerial floating image has a rectangular display area. In the background of the character in the image generated by the rendering, at least one vertex of the rectangle of the display screen of the image floating in the air is displayed in black, and is in a state that is invisible to the user in the air.

6. The mid-air floating image display device according to claim 4, wherein: The display screen of the aerial floating image has a rectangular display area. In the background of the character in the image generated by the rendering, at least one side of the rectangle of the display screen of the image floating in the air is included in a black display area and is not visible to the user in the air.

7. An air-suspended image display device, characterized in that: include: Image Processing Department; a display unit for displaying the image processed by the image processing unit; an optical system for generating an image suspended in mid-air based on the image displayed on the display unit; and An operation detection unit detects the operation performed by the user on the display screen of the air-floating image, wherein: The image processing unit performs image processing on an image generated by rendering a virtual 3D space in which an object having a character is arranged. The rendering of the virtual 3D space is performed by shooting with a virtual 3D space camera set in the virtual 3D space through perspective projection. The image generated by the rendering and displayed on the display unit is an image obtained by shooting the virtual 3D space in which the character is positioned, with the field of view of the virtual 3D space camera set so as to satisfy Lf ≥ 19788 / Pa, assuming that the focal length of the lens of the virtual 3D space camera converted to 35mm film is Lf and the diagonal length of the display screen of the suspended image is Pa. The units of Lf and Pa are mm.

8. The mid-air floating image display device according to claim 7, wherein: The display screen of the aerial floating image has a rectangular display area. In the background of the character in the image generated by the rendering, at least one vertex of the rectangle of the display screen of the image floating in the air is displayed in black, and is in a state that is invisible to the user in the air.

9. The mid-air floating image display device according to claim 7, wherein: The display screen of the aerial floating image has a rectangular display area. In the background of the character in the image generated by the rendering, at least one side of the rectangle of the display screen of the image floating in the air is included in a black display area and is not visible to the user in the air.

10. A character display device capable of communicating with a character, characterized in that: include: a display unit capable of displaying a character; speaker; microphone; Ministry of Communications; and Control unit, wherein The communication unit is capable of communicating with a server capable of executing reasoning of an artificial intelligence large-scale language model, sending prompt words containing natural language text information to the server, and receiving responses containing natural language text information from the server, The speaker outputs a natural language voice by synthesizing the voice so that the voice of the character displayed on the display unit sounds to the user based on the text information included in the answer.

11. The character display device according to claim 10, wherein: The display screen of the display unit is a display screen formed in an image suspended in the air.

12. The character display device according to claim 10, wherein: including a storage unit, The prompt words include setting prompt words containing natural language text information, wherein the setting prompt words can indicate the setting of at least one of the identity, name, conversation characteristics, and personality of the character to be played by the character using natural language text information. The storage unit saves and maintains a conversation history between a character and a user in a database in natural language, wherein the conversation history is generated based on natural language text information transmitted between the character display device and the server via the communication unit.

13. The character display device according to claim 12, wherein: After a series of sessions established by the character display device and the server transmitting natural language text information via the communication unit are terminated, when a new session is started by sending a setting prompt word containing natural language text information to the server via the communication unit again, the natural language text information based on the session history stored in the database will be included in the setting prompt word sent to the server.

14. The character display device according to claim 13, wherein: The database of the storage unit stores natural language text information resulting from summarizing the conversation history, thereby reducing the number of word units of the natural language text information based on the conversation history contained in the set prompt words sent to the server.

15. The character display device according to claim 14, wherein: The article summarization process is executed in a server different from the server, or is executed by an article summarization program loaded into a memory of the character display device.

16. The character display device according to claim 14, wherein: The conversation history recorded in the database of the storage unit is cleared from the oldest history, thereby reducing the number of word units of the natural language text information based on the conversation history contained in the set prompt words transmitted to the server.

17. The character display device according to claim 12, wherein: The control unit is capable of performing control to switch the character displayed on the display unit to a character different from the character displayed on the display unit among a plurality of characters. In the database maintained by the storage unit, for each of the multiple characters, a setting prompt word containing natural language text information indicating the setting of at least one of the identity, name, conversation characteristics or personality to be played by the character, and a conversation history between the character and the user generated based on the natural language text information transmitted by the character display device and the server via the communication unit are recorded in association.

18. The character display device according to claim 17, wherein: After a series of sessions established by transmitting natural language text information between the character display device and the server via the communication unit are terminated for a specified character via the communication unit, when a setting prompt word containing natural language text information is sent to the server via the communication unit again to start a new session for the specified character, a session history associated with the specified character is selected from the session histories associated with each of the multiple characters stored in the database, and the natural language text information based on the session history is included in the setting prompt word sent to the server.

19. The character display device according to claim 18, wherein: When the control unit controls the character to be switched to a character different from the character displayed on the display unit, the frequency of sending the setting prompt word to the server is reduced by the presence of the following state, wherein the state is: Even after the setting of the character to be displayed on the display unit has been switched to the image of the different character in switching to a character different from the character displayed on the display unit, and the setting of the synthetic sound to be output from the speaker has also been set to the synthetic sound corresponding to the different character, the setting prompt word including the natural language text information based on the conversation history stored in the database in association with the different character is not sent to the server.

20. The character display device according to claim 17, wherein: The display screen of the display unit is a display screen formed in an image suspended in the air.

Images