Aerial image display control device, method, and program

Abstract

An aerial image display control device according to one embodiment comprises: a generation unit that generates a video content related to an aerial image which is formed in the air by reflecting light by a reflection member and which is continuous from the vertical upper side to the vertical lower side across the reflection surface when viewed from a reflection surface of the reflection member; an effect superimposition unit that superimposes an effect on the video content generated by the generation unit such that, when the video content generated by the generation unit is displayed as the aerial image, an aerial image composed of a portion corresponding to an aerial image on the vertical upper side when viewed from the reflection surface of the reflection member and a portion corresponding to an aerial image on the vertical lower side when viewed from the reflection surface of the reflection member in the generated video content is perceived by a viewer as the aerial image which is continuous from the vertical upper side to the vertical lower side across the reflection surface of the reflection member when viewed from the reflection surface of the reflection member; and an aerial image display control unit that causes an aerial image display device to display, as the aerial image, the video content which has been generated by the generation unit and on which the effect has been superimposed by the effect superimposition unit.

Description

Aerial image display control device, method, and program

[0001] An embodiment of the present invention relates to an aerial image display control device, method, and program.

[0002] Aerial images are real images displayed in real space, and viewers can view them without wearing any device. For this reason, aerial images are used in a variety of applications, including Augmented Reality (AR).

[0003] Since aerial images have no substance, they can be displayed inside solid objects. For example, Non-Patent Document 1 proposes an aerial image display optical system that spans both the interior and exterior spaces of a specular reflecting surface.

[0004] Fumika Sano, Takamune Makiguchi, Aisui Hoshi, Hiroshi Chiaki, Takayoshi Mochizuki, "Proposal of an aerial imaging system that displays an aerial image and its reflected image on and within a horizontal surface with specular reflection characteristics," IEICE Technical Report, MVE2023-36 (2024)

[0005] In the optical system described above, a mid-air image is displayed on a specular reflective surface, but because the specular reflective surface has little information about the planar position of the texture, etc., it is difficult to recognize the relative positional relationship with the mid-air image displayed on it. Furthermore, it is difficult to recognize the type of material the reflective surface is made of just by looking at it.

[0006] This invention has been made in light of the above-mentioned circumstances, and its purpose is to provide an aerial image display control device, method, and program that can improve the visibility of aerial images that extend continuously from the vertically upper side to the vertically lower side across the reflective surface of the reflective member.

[0007] An aerial image display control device according to one aspect of the present invention comprises: a generation unit that generates video content relating to an aerial image formed in the air by reflecting light off a reflecting member, the aerial image extending continuously from a vertically upper side as viewed from the reflecting surface of the reflecting member to a vertically lower side across the reflecting surface; an effect superimposition unit that superimposes an effect on the video content generated by the generation unit so that, when the video content generated by the generation unit is displayed as an aerial image, an aerial image consisting of a portion of the generated video content corresponding to the aerial image vertically upper side as viewed from the reflecting surface of the reflecting member and a portion of the generated video content corresponding to the aerial image vertically lower side as viewed from the reflecting surface of the reflecting member is perceived by an observer as an aerial image extending continuously from a vertically upper side as viewed from the reflecting surface of the reflecting member to a vertically lower side across the reflecting surface; and an aerial image display control unit that causes an aerial image display device to display the video content generated by the generation unit, with the effect superimposed by the effect superimposition unit, as an aerial image.

[0008] An aerial image display control method according to one aspect of the present invention is a method performed by an aerial image display control device, the method comprising: generating, by a generation unit of the aerial image display control device, video content relating to an aerial image formed in the air by reflecting light off a reflecting member, the aerial image extending vertically from above as viewed from the reflecting surface of the reflecting member to below as viewed from the reflecting surface; superimposing, by an effect superimposing unit of the aerial image display control device, an effect on the video content generated by the generation unit so that, when the video content generated by the generation unit is displayed as an aerial image, an aerial image consisting of a portion of the generated video content corresponding to the aerial image vertically above as viewed from the reflecting surface of the reflecting member and a portion of the generated video content corresponding to the aerial image vertically below as viewed from the reflecting surface of the reflecting member is perceived by an observer as an aerial image extending vertically from above as viewed from the reflecting surface of the reflecting member to below as viewed from the reflecting surface of the reflecting member; and causing, by an aerial image display control unit of the aerial image display control device, the video content generated by the generation unit upon which the effect has been superimposed by the effect superimposition unit as an aerial image.

[0009] According to the present invention, it is possible to improve the visibility of an aerial image that continues from the vertically upper side as viewed from the reflecting surface of the reflecting member to the vertically lower side across the reflecting surface.

[0010] FIG. 1 is a block diagram showing the overall configuration of an aerial image display system according to an embodiment of the present invention. FIG. 2 is a plan view showing an example of the basic configuration of an aerial image display device. FIG. 3 is a plan view showing an example of an imaging optical path of an aerial image displayed by the aerial image display device. FIG. 4 is a plan view showing an example of an aerial image display device applied to the aerial image display system according to this embodiment. FIG. 5 is a plan view showing an example of an imaging optical path of an aerial image displayed by the aerial image display device applied to the aerial image display system according to this embodiment. FIG. 6 is a diagram showing an application example of a control device for the aerial image display system according to an embodiment of the present invention. FIG. 7 is a flowchart showing an example of a processing operation procedure by the control device of the aerial image display system. FIG. 8 is a diagram showing an example of content generated by a content generation unit. FIG. 9 is a diagram showing an example of content on which an effect is superimposed by an effect control unit. FIG. 10 is a diagram showing an example of an aerial image whose transparency is changed depending on the depth. FIG. 11 is a diagram showing an example of an effect superimposed on content by an effect superimposition unit. FIG. 12 is a diagram showing an example of content displayed by the aerial image display device.

[0011] Hereinafter, several embodiments will be described with reference to the drawings. In the following description, components having the same functions and configurations will be given the same reference numerals.

[0012] 1.1 Configuration of the Aerial Image Display System Fig. 1 is a block diagram showing the overall configuration of an aerial image display system according to one embodiment of the present invention. As shown in Fig. 1, the aerial image display system 5 includes an aerial image display device 1, a reflecting member 2, and a control device 9. The aerial image display device 1 displays an aerial image. For example, the aerial image display device 1 can display an image in the air on the reflecting member 2. Details of the configurations of the aerial image display device 1 and the reflecting member 2 will be described later.

[0013] The control device 9 is an aerial image display control device that controls the aerial image display device 1. The control device 9 includes a processor 90, a read only memory (ROM) 91, a random access memory (RAM) 92, a storage 93, and an interface 94.

[0014] The processor 90 is a processing circuit capable of executing various programs. The processor 90 controls the overall operation of the system including the aerial image display device 1. The processor 90 includes a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit). Note that multiple processors 90 may be provided within the control device 9.

[0015] The ROM 91 is a non-volatile semiconductor memory such as an EEPROM (registered trademark), and stores programs, control data, and the like for controlling the aerial image display device 1.

[0016] The RAM 92 is a volatile semiconductor memory such as a dynamic RAM (DRAM) or a static RAM (SRAM). The RAM 92 is used as a working area for the processor 90.

[0017] The storage 93 is a non-volatile storage device such as a hard disc drive (HDD), a solid state drive (SSD), or a memory card. The storage 93 stores various types of information and various types of data. Programs and control data may be stored in the storage 93. The storage 93 stores, for example, image data to be projected onto the aerial image display device 1.

[0018] The interface 94 includes various connectors, various ports, a signal processing circuit, a communication module, etc. The interface 94 connects the control device 9 to the aerial image display device 1. The interface 94 is responsible for inputting and outputting information and data, various processes on information and data, and various controls for acquiring information and data. The interface 94 accepts input from a user of the aerial image display device 1 via input devices (not shown), such as a touch panel, a keyboard, a mouse, operation buttons, and a microphone. The interface 94 provides various information and data to a user of the aerial image display device 1 via output devices (not shown), such as a display and a speaker. The interface 94 can transfer information and data between the aerial image display device 1 and the control device 9 via communication via a wireless or wired network. The interface 94 sends various control signals from the processor 90 to the aerial image display device 1. The interface 94 sends various control signals from the aerial image display device 1 to the processor 90.

[0019] The aerial image display device 1 described below is a device equipped with a real image optical system configured to display a real aerial image on a reflecting member 2 based on a display image displayed on the display surface of a display. The real aerial image displayed by the aerial image display device 1 is displayed in a direction approximately perpendicular to the reflecting surface of the reflecting member 2. For convenience of explanation, the plane parallel to the reflecting surface of the reflecting member 2 provided in the aerial image display system 5 is referred to as the XY plane. The plane parallel to the display surface of the real aerial image is referred to as the YZ plane. The direction in which the XY plane and the YZ plane intersect is referred to as the Y direction. The direction perpendicular to the Y direction in the XY plane is referred to as the X direction. The direction perpendicular to the Y direction in the YZ plane is referred to as the Z direction. The user's line of sight LOS is assumed to coincide with the direction obtained by combining the +X direction and the -Z direction. The opposite direction of the user's line of sight LOS, i.e., the direction approaching the user, is assumed to coincide with the direction obtained by combining the -X direction and the +Z direction.

[0020] FIG. 2 is a plan view showing an example of the basic configuration of an aerial image display device. The aerial image display device 1A shown in FIG. 2 is not the aerial image display device applied to the aerial image display system according to this embodiment, but is shown to facilitate understanding of the optical system of the aerial image display device. In this example, the aerial image display device is referred to as the aerial image display device 1A to distinguish it from the aerial image display system according to this embodiment. FIG. 2 shows the planar positional relationship between the reflecting member 2 and the aerial image display device 1A when viewed in the Y direction (in the ZX plane as viewed from the -Y side).

[0021] The aerial image display device 1A is placed on a reflecting member 2. The reflecting member 2 has a reflecting surface configured to reflect incident light. The reflecting surface of the reflecting member 2 includes a mirror, a glossy table surface, etc. The reflecting member 2 includes, for example, a non-glossy surface on which a half mirror or glossy film is placed. The reflecting member 2 includes, for example, a non-glossy surface on which a glossing agent is applied.

[0022] The reflecting member 2 may have a reflective polarizing film attached to its reflective surface. By attaching a reflective polarizing film to the reflecting member 2, for example, it is possible to improve the reflection efficiency of the reflective surface. The reflecting member 2 may also be included in the aerial image display device 1. That is, the aerial image display device 1A may be equipped with a reflective member 2 such as a half mirror.

[0023] The aerial image display device 1A includes a display 10, a reflective polarizing film 11, a beam splitter 12, a wave plate 13, and a reflector 14.

[0024] The beam splitter 12 is, for example, a polarizing beam splitter. The beam splitter 12 transmits light of the same polarization state and reflects light of the same polarization state. In the following, the polarization state of the light that transmits the beam splitter 12 is assumed to be S-polarized, and the polarization state of the light that is reflected by the beam splitter 12 is assumed to be P-polarized.

[0025] The beam splitter 12 is disposed inside the aerial image display device 1A at an arbitrary angle α in the negative direction (clockwise) with respect to the reflecting surface of the reflecting member 2 when viewed in the Y direction (in the ZX plane viewed from the -Y side). When viewed in the Y direction, one end of the beam splitter 12 intersects with one end of the reflecting member 2 or an extension thereof toward the inside of the aerial image display device 1A. The intersection point between the beam splitter 12 and the reflecting member 2 is sometimes referred to as the starting point A1. The reflecting surface of the beam splitter 12 is disposed so as to face the inside of the aerial image display device 1A. The angle α will be described later.

[0026] Reflective polarizing film 11, for example, reflects P-polarized light and transmits S-polarized light. Reflective polarizing film 11 is attached to the reflective surface of beam splitter 12. The reflective surface of reflective polarizing film 11 is positioned so as to face the inside of aerial image display device 1A.

[0027] The reflector 14 has a reflective surface configured to retroreflect incident light. The reflective surface of the reflector 14 is positioned so as to face the inside of the aerial image display device 1A. The reflector 14 is positioned inside the aerial image display device 1A, tilted in the positive direction (counterclockwise) by an arbitrary angle β with respect to the reflective surface of the reflective polarizing film 11 when viewed in the Y direction. When viewed in the Y direction, one end of the reflector 14 intersects with the other end of the beam splitter 12 (the end opposite the starting point A1) or its extension. The intersection point between the reflector 14 and the beam splitter 12 is sometimes referred to as the starting point B1. The angle β will be described later.

[0028] The wave plate 13 is, for example, a λ / 4 wave plate. The wave plate 13 is configured to output incident light with a phase shift of π / 2 between P-polarized light and S-polarized light. The wave plate 13 is attached to the reflective surface of the reflector 14. In other words, the wave plate 13 is disposed so as to face the reflective surface of the reflective polarizing film 11.

[0029] The display 10 is, for example, a polarized display, and emits, for example, P-polarized light.

[0030] The display 10 is disposed so that its display surface faces the inside of the aerial image display device 1A. When viewed in the Y direction, one end of the display 10 is disposed so as to intersect with the other end of the reflector 14 (the end opposite the starting point B1). In the example shown in Fig. 2, the intersection of the display 10 and the reflector 14 may be referred to as starting point C1. The position of the other end of the display 10 (the end opposite the starting point C1) may be referred to as starting point D1.

[0031] The distance between the starting point A1 and the starting point C1 has a length H (H is a positive real number). For example, when viewed in the Y direction, the display 10 is disposed tilted in the positive direction (counterclockwise) with respect to the line segment connecting the starting point A1 and the starting point C1. When viewed in the Y direction, the angle that the display 10 forms with respect to the wave plate 13 in the positive direction (counterclockwise) may be referred to as an arbitrary angle γ.

[0032] The orientations of display 10 and beam splitter 12 are adjusted so that P-polarized light emitted from display 10 is incident on the reflective surface of reflective polarizing film 11. Similarly, the orientations of beam splitter 12 and reflector 14 are adjusted so that P-polarized light reflected by reflective polarizing film 11 is incident on the reflective surface of reflector 14. That is, angles α, β, and γ can be collectively adjusted so that the P-polarized light emitted from display 10 follows an optical path that ultimately forms a real aerial image. Furthermore, angles α, β, and γ can be collectively adjusted so that the P-polarized light emitted from display 10 follows an optical path that efficiently forms a real aerial image (with high brightness efficiency). In addition, the position of origin D1, which is the other end of display 10, can be adjusted so that it is located on the +Z side of the extension of the reflective surface of reflecting member 2.

[0033] 1.2 Optical Path of Aerial Image Fig. 3 is a plan view showing an example of an optical path of an aerial image displayed by the aerial image display device. As shown in Fig. 3, the aerial image display device 1A can display an aerial image AI1 in the line of sight direction LOS by displaying a display image DI1 on the display surface of the display 10.

[0034] The display 10 emits P-polarized light (hereinafter referred to as emitted light Pa1) from the display image DI1. The emitted light Pa1 is totally reflected by the reflective surface of the reflective polarizing film 11. The totally reflected light is referred to as reflected light Pb1. The reflected light Pb1 is reflected toward the reflector 14.

[0035] The reflected light Pb1 is retroreflected by the reflector 14 via the wave plate 13. The reflected light Pb1 passes through the wave plate 13 twice during the retroreflection process. The reflected light Pb1 that has passed through the wave plate 13 twice is referred to as reflected light Sc1. The phase of the reflected light Sc1 is shifted by π by retroreflection via the wave plate 13, and the reflected light becomes S-polarized light.

[0036] The reflected light Sc1 is retroreflected toward the beam splitter 12. The reflected light Sc1 is transmitted through the reflective polarizing film 11 and the beam splitter 12. The transmitted reflected light Sc1 is referred to as transmitted light Sd1.

[0037] The transmitted light Sd1 is reflected by the reflecting surface of the reflecting member 2. The reflected transmitted light Sd1 is referred to as reflected light Se1. The reflected light Se1 forms an aerial image AI1 and displays it in the line of sight LOS. Here, the distance between the aerial image AI1 and the starting point A1 is a length H.

[0038] 1.3 Effects According to the aerial image display device 1A described above, it is possible to remove transmitted light from the display 10 while maintaining luminance efficiency.

[0039] As described above, in the aerial image display device 1A, the display 10 is configured to emit P-polarized light, and the reflective polarizing film 11 and beam splitter 12 are configured to transmit S-polarized light and reflect P-polarized light. In this configuration, theoretically, 100% of the emitted light Pa1 becomes reflected light Pb1. In other words, because the emitted light Pa1 does not contain any components (S-polarized light) that attempt to pass through the beam splitter 12 and reflective polarizing film 11, theoretically, 100% of the emitted light becomes reflected light Pb1.

[0040] Next, the aerial image display device 1A includes a wave plate 13 between the reflective polarizing film 11 and the reflector 14, and the wave plate 13 is a λ / 4 wave plate. Therefore, when the reflector 14 performs total reflection, theoretically, 100% of the reflected light Pb1 becomes reflected light Sc1. Theoretically, 100% of the reflected light Sc1, which has become S-polarized, becomes transmitted light Sd1 and passes through the reflective polarizing film 11 and the beam splitter 12. In other words, because the reflected light Sc1 does not contain the component (P-polarized light) that is intended to be reflected by the beam splitter 12 and the reflective polarizing film 11, theoretically, 100% becomes transmitted light Sd1.

[0041] Furthermore, if the reflectance of the reflecting surface of the reflecting member 2 is 100%, theoretically 100% of the transmitted light Sd1 becomes reflected light Se1.

[0042] In this way, in the aerial image display device 1A, if a material with a reflectance of 100% is used for the reflective surface of the reflective member 2, the aerial image AI1 can theoretically be formed with a brightness of 100%.

[0043] In the aerial image display device 1A, a polarized display is used for the display 10, thereby removing from the output light Pa1 the component (S-polarized light) that attempts to pass through the beam splitter 12 and the reflective polarizing film 11. Because the output light Pa1 does not contain any component that attempts to pass through the beam splitter 12 and the reflective polarizing film 11, there is no need to place a field of view control film or the like on the optical path to remove the transmitted light. By not placing a field of view control film or the like on the optical path, the aerial image display device 1A can maintain the brightness and image quality of the aerial image. In other words, the aerial image display device 1A can remove the transmitted light from the display 10 while maintaining brightness efficiency. As a result, the aerial image display device 1A can improve the user's visibility.

[0044] Furthermore, by adjusting the angle γ, the aerial image display device 1A can prevent light emitted from the display 10 from directly entering the reflector 14. Therefore, the aerial image display device 1A can be designed so that, in theory, stray light from the display 10 is not generated.

[0045] As described above, the aerial image display device 1A uses a reflector and a beam splitter, but does not use retro-transmitting optical elements such as Micro-Mirror Array Plates (MMAPs), etc. This makes it relatively easy to increase the size of the device, and therefore makes it possible to increase the size of the aerial image.

[0046] Furthermore, the aerial image display device 1A can adjust the position of the origin D1, which is the other end of the display 10, to the +Z side of the extension of the reflective surface of the reflecting member 2. Therefore, the aerial image display device 1A can be used simply by placing it on the reflective surface of an existing reflecting member 2. In other words, the optical system of the aerial image display system 5 can be completed on a tabletop.

[0047] Next, an aerial image display device 1 that is applied to the aerial image display system according to this embodiment will be described, focusing on the configuration that differs from that shown in FIG.

[0048] 2.1 Configuration of the aerial image display system The configuration of the aerial image display device 1 applied to the aerial image display system of this embodiment differs from the configuration shown in Figure 2 in that it can display an aerial image on the -Z direction side of the reflective surface of the reflective member 2.

[0049] Specifically, the aerial image display system 5 shown in FIG. 4 differs from the aerial image display system 5 shown in FIG. 2 mainly in the structure of the display 10.

[0050] 4 is a plan view showing an example of an aerial image display device applied to the aerial image display system according to this embodiment. The aerial image display device 1 shown in FIG. 4 includes a display 10, a reflective polarizing film 11, a beam splitter 12, a wave plate 13, and a reflector 14.

[0051] The display 10 is, for example, a polarized display. For example, P-polarized light is emitted from the display 10. The display 10 is disposed so that the display surface faces the inside of the aerial image display device 1.

[0052] When viewed in the Y direction, the display 10 has a structure in which the display 10A shown in Fig. 2 is extended toward the reflector 14. In the display 10, the portion that is extended more than the display 10A shown in Fig. 2 is hereinafter referred to as a display extension portion 102. Hereinafter, the portion of the display 10 shown in Fig. 4 that is the same as the display 10 shown in Fig. 2 is referred to as a display portion 101. In other words, the display 10 shown in Fig. 4 has the display portion 101 and the display extension portion 102.

[0053] Here, the coordinate positions of the starting points C1 and D1 are the same as in the example shown in Fig. 2. Therefore, when viewed in the Y direction, the starting point C1 corresponds to the boundary between the display portion 101 and the display extension portion 102. When viewed in the Y direction, the starting point D1 corresponds to the end of the display 10 on the display portion 101 side.

[0054] For example, the display 10 is disposed tilted in the positive direction (counterclockwise) with respect to the line segment connecting the starting point A1 and the starting point C1 when viewed in the Y direction. The angle that the display 10 forms with respect to the wave plate 13 in the positive direction (counterclockwise) when viewed in the Y direction is referred to as an arbitrary angle γ, as in the example shown in FIG.

[0055] The size of the reflector 14 may be adjusted as the display 10 is expanded compared to the display 10 shown in FIG.

[0056] 2.2 Optical Path of Aerial Image Fig. 5 is a plan view showing an example of an optical path of an aerial image displayed by the aerial image display device applied to the aerial image display system according to this embodiment. As shown in Fig. 5, the aerial image display device 1 displays a display image DI2 on the display surface of the display 10, thereby displaying an aerial image AI2 in the line of sight LOS.

[0057] The aerial image AI2 has, for example, an aerial image AI21 which is a portion displayed on the +Z direction side of the reflecting surface, and an aerial image AI22 which is a portion displayed on the -Z direction side of the reflecting surface of the reflecting member 2.

[0058] The display 10 emits output light Pa22 from the display surface of the display extension portion 102, and emits output light Pa21 from the display surface of the display portion 101. The output light Pa21 follows the same optical path as the output light Pa1 shown in FIG. 3 and forms an image in the same manner. The output light Pa21 ultimately forms an image on the +Z direction side of the reflecting surface of the reflecting member 2. That is, an aerial image AI21 is formed from the output light Pa21, and the aerial image AI21 is an aerial image similar to the aerial image AI1.

[0059] The light Pa22 emitted from the display extension 102 is P-polarized light. The light Pa22 is totally reflected by the reflective surface of the reflective polarizing film 11. The totally reflected light is called reflected light Pb2. The reflected light Pb2 is reflected in the direction of the reflector 14.

[0060] The reflected light Pb2 is retroreflected by the reflector 14 via the wave plate 13. The reflected light Pb2 passes through the wave plate 13 twice during the retroreflection process. The reflected light Pb2 that has passed through the wave plate 13 twice is referred to as reflected light Sc2. The phase of the reflected light Sc2 is shifted by π by retroreflection via the wave plate 13, and it becomes S-polarized light.

[0061] The reflected light Sc2 is retroreflected toward the beam splitter 12. The reflected light Sc2 is transmitted through the reflective polarizing film 11 and the beam splitter 12. The transmitted reflected light Sc2 is referred to as transmitted light Sd2.

[0062] The transmitted light Sd2 is reflected by the reflecting surface of the reflecting member 2. The reflected transmitted light Sd2 is referred to as reflected light Se2.

[0063] The position in the X direction where the reflected light Se2 is reflected by the reflecting surface is on the -X side (away from the origin A1) of the position where the aerial image AI21 is formed. Therefore, the reflected light Se2 forms an aerial image AI22 on the -Z side of the reflecting surface of the reflecting member 2. That is, the reflected light Se2 forms an aerial image AI22 and displays it in the line of sight LOS. Here, there is a length H between the aerial image AI1 and the origin A1.

[0064] 2.3 Effects According to the aerial image display device 1 described above, an aerial image can be displayed also on the −Z direction side of the reflective surface of the reflecting member 2. As described above, in the aerial image display device 1, the display 10 has a display extension section 102. The display extension section 102 is located closer to the reflector 14 (in the direction away from the starting point D1) than the line segment connecting the starting points A1 and C1.

[0065] Therefore, the emitted light Pa22 from the display surface of the display extension unit 102 ultimately forms an aerial image AI22 on the −Z side of the reflective surface of the reflecting member 2. This is because the position at which the reflected light Se2 is reflected on the reflective surface of the reflecting member 2 is on the −X side (a direction away from the origin A1) of the position at which the aerial image AI21 is formed. In this way, the aerial image display device 1 can display an aerial image on the −Z side in addition to the aerial image on the +Z side of the reflective surface of the reflecting member 2.

[0066] Furthermore, similar to the example shown in FIG. 2 , the aerial image display device 1 uses a polarized display for the display 10, eliminating the need to place a field of view control film or the like in the optical path to remove transmitted light from the display 10. Because a field of view control film or the like is not placed in the optical path, the aerial image display device 1 can maintain the brightness and image quality of the aerial image. That is, the aerial image display device 1 can remove transmitted light from the display 10 while maintaining brightness efficiency. In the aerial image display device 1, similar to the aerial image display device 1A shown in FIG. 2 , if a material with a reflectivity of 100% is used for the reflective surface of the reflective member 2, the aerial image AI2 can theoretically be formed with 100% brightness. Therefore, the aerial image display device 1 can improve user visibility.

[0067] 2, the aerial image display device 1 can adjust the angle γ so that the light emitted from the display 10 does not directly enter the reflector 14. Therefore, the aerial image display device 1 can be designed so that, in theory, stray light from the display 10 is not generated.

[0068] 2, the aerial image display device 1 uses a reflector and a beam splitter, but does not use a retro-transmitting optical element such as an MMAP, etc. Therefore, it is relatively easy to increase the size of the device, and it is possible to increase the size of the aerial image.

[0069] 2, the aerial image display device 1 can adjust the position of the origin D1, which is the other end of the display 10, to the +Z side of the extension of the reflecting surface of the reflecting member 2. Therefore, the aerial image display device 1 can be used simply by placing it on the reflecting surface of an existing reflecting member 2. In other words, the optical system of the aerial image display system 5 can be completed on a tabletop. The configurations shown in FIGS. 1 to 5 are also described in International Application PCT / JP2023 / 035260.

[0070] In this embodiment, as described above, in order to distinguish the appearance of the aerial image displayed on the +Z side of the reflective surface of the reflecting member 2 and the aerial image displayed on the -Z side of the reflective surface of the reflecting member 2 from the aerial image displayed on the +Z side of the reflective surface of the reflecting member 2 by the aerial image display device 1 shown in Figure 4 from the aerial image displayed on the -Z side of the reflective surface of the reflecting member 2, a transparency effect is applied to the aerial image displayed on the -Z side of the reflective surface of the reflecting member 2.

[0071] This makes it easier to recognize the relative positions of the aerial image displayed on the +Z side of the reflective surface of the reflective member 2 and the aerial image displayed on the −Z side of the reflective surface of the reflective member 2 relative to the reflective member 2, making it easier for the user to visually recognize that the aerial image is also displayed on the −Z side of the reflective surface of the reflective member 2, and controls the reflective surface of the reflective member 2 to be more impressive to the user, and to give the user the impression that the reflective surface of the reflective member 2 is made of a material different from its original material.

[0072] By performing such control, the user can visually distinguish between the aerial image displayed on the +Z side of the reflective surface of the reflecting member 2 and the aerial image displayed on the -Z side of the reflective surface of the reflecting member 2.

[0073] Fig. 6 is a diagram showing an application example of a control device of an aerial image display system according to an embodiment of the present invention. Fig. 7 is a flowchart showing an example of a procedure of processing operations by the control device of the aerial image display system. Fig. 8 is a diagram showing an example of content generated by a content generation unit. Fig. 9 is a diagram showing an example of content on which an effect has been superimposed by an effect control unit. As shown in Fig. 6, the control device 9 has a content generation unit 110, an effect control unit 120, and an aerial image display control unit 130.

[0074] The effect control unit 120 has a material information input unit 121 , a parameter setting unit 122 , a parameter value database 123 , and an effect superimposition unit 124 .

[0075] The content generator 110 generates content as an aerial image displayed on the +Z direction side and the −Z direction side of the reflecting surface of the reflecting member 2, for example as shown in FIG. 8 (S10).

[0076] The effect control unit 120 creates a transparent expression effect and superimposes a transparent effect, such as that shown in Figure 9, on the part of the content generated by the content generation unit 110 that corresponds to the aerial image displayed on the -Z direction side of the reflective surface of the reflective member 2.

[0077] The material information input unit 121 of the effect control unit 120 receives input relating to the selection of a reflective surface and an expression that the user wants to perceive from the reflective surface (S20).

[0078] The parameter setting section 122 of the effect control section 120 acquires the values ​​of the parameters that control the reproduction of the expression input by the material information input section 121 from the parameter value database 123 (S30).

[0079] The parameters include, for example, the following (1) to (7): (1) Texture information (appearance): Parameters that control the texture of the reflective surface, such as the pattern and color of the reflective surface, glossiness, and virtual transparency, so that it changes, for example, as it goes further back in the −Z direction from the reflective surface of the reflecting member 2. (2) Texture information (movement): Parameters that control whether or not the texture moves.

[0080] For example, if the texture has movement, the "texture information (movement)" is a parameter that controls the speed and direction of movement of the texture.

[0081] (3) Refractive index: A parameter that controls the appearance by setting a virtual refractive index on the reflective surface. (4) Transparency according to depth: A parameter that controls the appearance of the content so that it appears darker as it moves further back in the −Z direction from the reflective surface of the reflective member 2.

[0082] (5) Blur intensity according to depth: A parameter that controls the representation so that the blur intensity of the content increases as it moves further back from the reflective surface of the reflecting member 2 in the -Z direction. (6) Pixel size according to depth: A parameter that controls the representation so that the resolution of the content decreases as it moves further back from the reflective surface of the reflecting member 2 in the -Z direction. (7) Boundary and surface information: A parameter that assigns width and color to the boundary and boundary surfaces on the +Z side and the -Z side of the reflective surface to the aerial image according to the position of the reflective surface.

[0083] In the above description, the parameter values ​​are taken from the parameter value database 123, but manual adjustments may be made to these taken values, or all of the values ​​may be determined by the user.

[0084] The parameter value database 123 of the effect control unit 120 acquires and records in advance the material of the reflective surface and the parameters of the desired perceived expression of the reflective surface. The recorded parameter values ​​may be actual measurements or virtual values ​​set by the user.

[0085] Fig. 10 is a diagram showing an example of an aerial image in which transparency changes depending on depth. Fig. 10 shows an example in which the transparency of the portion of the content on the -Z side of the reflective surface of the reflective member 2 gradually changes as the image moves further back in the -Z direction than the reflective surface of the reflective member 2. It is also possible to represent the reflective surface in an indistinct manner by not showing the boundary between the +Z side and the -Z side of the reflective surface.

[0086] The effect superimposing section 124 of the effect control section 120 superimposes the effect set by the parameter setting section 122 onto the content (S40).

[0087] 11 is a diagram showing an example of an effect superimposed on content by the effect superimposition unit 124. When the effect superimposition unit 124 superimposes an effect on content, the effect superimposition unit 124 superimposes the effect vertically on the content (symbol a in (a) of FIG. 11 ) and superimposes the effect parallel to the content (symbol a in (a) of FIG. 11 ).

[0088] In addition, the effect control unit 120 may execute all processing of each unit before the aerial image is displayed on the aerial image display device 1, or may execute processing of each unit sequentially while the aerial image is being displayed on the aerial image display device 1.

[0089] Fig. 12 is a diagram showing an example of content displayed by the aerial image display device. The aerial image display control unit 130 controls the display of an aerial image on the reflective surface of the reflecting member 2 by the aerial image display device 1, for example, as shown in Fig. 12, using content generated by the content generation unit 110 and on which effects are superimposed by the effect control unit 120, for example, as shown in Fig. 9 (S40).

[0090] The methods described in each embodiment can be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, MO, etc.), or semiconductor memory (ROM, RAM, flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only execution programs but also tables or data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-mentioned processing by controlling the operation of this software means. The term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network.

[0091] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0092] DESCRIPTION OF SYMBOLS 1, 1A...Aerial image display device 2...Reflective member 5...Aerial image display system 9...Control device 10...Display 11...Reflective polarizing film 12...Beam splitter 13...Wave plate 14...Reflector 90...Processor 91...ROM 92...RAM 93...Storage 94...Interface 101...Display section 102...Display extension section 111...Content generation section 120...Effect control section 121...Material information input section 122...Parameter setting section 123...Parameter value database 124...Effect superimposition section 130...Aerial image display control section A1, B1, C1, D1...Origin AI...Aerial image DI...Displayed image Pa...Emitted light Pb, Sc, Se...Reflected light Sd...Transmitted light

Claims

1. An aerial image display control device comprising: a generation unit that generates video content relating to an aerial image formed in the air by reflecting light off a reflecting member, the aerial image extending from a vertically upper side as viewed from the reflecting surface of the reflecting member to a vertically lower side across the reflecting surface; an effect superimposition unit that superimposes effects on the video content generated by the generation unit so that, when the video content generated by the generation unit is displayed as an aerial image, an aerial image consisting of a portion of the generated video content corresponding to the aerial image vertically upper side as viewed from the reflecting surface of the reflecting member and a portion corresponding to the aerial image vertically lower side as viewed from the reflecting surface of the reflecting member is perceived by an observer as an aerial image extending from a vertically upper side as viewed from the reflecting surface of the reflecting member to a vertically lower side across the reflecting surface; and an aerial image display control unit that causes an aerial image display device to display the video content generated by the generation unit, upon which the effect has been superimposed by the effect superimposition unit, as an aerial image.

2. An aerial image display control device as described in claim 1, wherein the effect superimposition unit superimposes an effect on a portion of the video content generated by the generation unit that corresponds to an aerial image vertically below the reflective surface of the reflective member.

3. The aerial image display control device of claim 1, wherein the effect superimposition unit superimposes one or more of the following effects on the video content generated by the generation unit: (1) an effect that causes the video content to appear darker as it moves vertically downward from the reflective surface of the reflecting member; (2) an effect that causes the video content to become increasingly blurred as it moves vertically downward from the reflective surface of the reflecting member; and (3) an effect that causes the texture to change as it moves vertically downward from the reflective surface of the reflecting member.

4. A method performed by an aerial image display control device, comprising: generating, by a generation unit of the aerial image display control device, video content relating to an aerial image formed in the air by reflecting light off a reflecting member, the continuous aerial image extending from vertically above as viewed from the reflective surface of the reflecting member to vertically below as viewed from the reflective surface; superimposing, by an effect superimposing unit of the aerial image display control device, an effect on the video content generated by the generation unit so that, when the video content generated by the generation unit is displayed as an aerial image, an aerial image consisting of a portion of the generated video content corresponding to the aerial image vertically above as viewed from the reflective surface of the reflecting member and a portion corresponding to the aerial image vertically below as viewed from the reflective surface of the reflecting member is perceived by an observer as a continuous aerial image extending from vertically above as viewed from the reflective surface of the reflecting member to vertically below as viewed from the reflective surface of the reflecting member; and displaying, by an aerial image display control unit of the aerial image display control device, the video content generated by the generation unit onto which the effect has been superimposed by the effect superimposing unit as an aerial image.

5. An aerial image display control processing program that causes a processor to function as each part of the aerial image display control device according to any one of claims 1 to 3.

Images