Aerial image display device, aerial image display method, and aerial image display program

Abstract

An aerial image display device according to one embodiment comprises: a storage unit that stores the reflectance of a specular reflection surface; a reflection image brightness calculation unit that, on the basis of the reflectance, determines the brightness of an aerial image on the specular reflection surface to be displayed on the specular reflection surface and the brightness for displaying an aerial image in the specular reflection surface to be displayed in the specular reflection surface; and a display control unit that controls an optical system such that the aerial image on the specular reflection surface and the aerial image in the specular reflection surface are displayed using the determined brightnesses.

Description

Aerial image display device, aerial image display method, and aerial image display program

[0001] The present invention relates to an aerial image display device, an aerial image display method, and an aerial image display program.

[0002] Aerial image display technology has been proposed that displays aerial images in empty space, making objects appear as if they are actually present. Aerial images are real images displayed in real space. Viewers can view aerial images without wearing any device. For this reason, aerial images are used in a variety of applications, including augmented reality (AR).

[0003] Furthermore, methods for enhancing the sense of reality of aerial images have been studied in order to display the aerial images as if they actually exist in real space. For example, Non-Patent Document 1 proposes a method for using a projector to project the shadow of the aerial image onto a horizontal surface such as a desk on which the aerial image is displayed.

[0004] Kim et al., Mario: Mid-air augmented reality interaction with objects, 2014

[0005] Even when a shadow is projected onto a specular reflective surface using a projector, the projected light is reflected. Therefore, as an alternative to shadow projection, a reflected image of the aerial image can be displayed within the specular reflective surface, enhancing the sense of realism of the aerial image. However, the reflectivity of the specular reflective surface varies depending on the material and reflection angle. Therefore, in order to make the reflection of the aerial image appear similar to the reflection of the real object on various specular reflective surfaces, it is necessary to reproduce it in accordance with each material and reflection angle.

[0006] This invention was made in light of the above circumstances, and its purpose is to propose a technology that makes it possible to reproduce the reflected image of an aerial image displayed on a plane with various specular reflection characteristics in accordance with the reflectance of each plane.

[0007] In order to solve the above problem, one embodiment of the aerial image display device of the present invention comprises a memory unit that stores the reflectance of a specular reflecting surface, a reflected image brightness calculation unit that determines the brightness of an aerial image on the specular reflecting surface to be displayed on top of the specular reflecting surface and the brightness for displaying an aerial image within the specular reflecting surface to be displayed within the specular reflecting surface based on the reflectance, and a display control unit that controls the optical system to display the aerial image on the specular reflecting surface and the aerial image within the specular reflecting surface using the determined brightness.

[0008] According to one aspect of the present invention, it is possible to reproduce a reflected image of an aerial image displayed on a plane having various specular reflection characteristics in accordance with the reflectance of each plane.

[0009] FIG. 1 is a diagram showing a schematic configuration of an aerial image display system according to a first embodiment. FIG. 2 is a block diagram showing an example of the hardware and software configurations of the aerial image display device according to the first embodiment. FIG. 3 is a diagram showing an example of a light source according to the first embodiment. FIG. 4 is a diagram showing an example of a reference light source and a luminance meter when measuring a luminance measurement value R according to the first embodiment. FIG. 5 is a diagram showing an example of a reference light source and a luminance meter when measuring a reflecting surface measurement value RR according to the first embodiment. FIG. 6 is a flowchart showing an example of an aerial image display operation by the aerial image display device according to the first embodiment. FIG. 7 is a diagram showing an example of display of an aerial image on a reflecting surface and an aerial image in a reflecting surface according to the first embodiment. FIG. 8 is a diagram showing an example of measuring the luminance of an aerial image on a reflecting surface and an aerial image in a reflecting surface according to the second embodiment. FIG. 9 is a diagram showing an example of measuring the luminance of a reference light source on a specular reflecting surface and the luminance of a reference light source that has passed through a specular reflecting surface according to the second embodiment. FIG. 10 is a flowchart showing an example of an aerial image display operation by the aerial image display device according to the second embodiment. FIG. 11 is a diagram showing an example of display of an aerial image on a reflecting surface and an aerial image in a reflecting surface according to the second embodiment.

[0010] The aerial image display device, the aerial image display method, and the aerial image display program will be described in detail below with reference to the drawings. In the following embodiments, parts with the same numbers perform similar operations, and redundant description will be omitted. For example, when there are multiple identical or similar elements, a common symbol may be used to describe each element without distinguishing between them, or a subnumber may be used in addition to the common symbol to describe each element with distinction between them.

[0011] (Aerial Image Display System) First, the schematic configuration of the aerial image display system will be described. FIG. 1 is a diagram showing the schematic configuration of an aerial image display system according to a first embodiment. As shown in FIG. 1, the aerial image display system 100 generates an on-reflecting-surface aerial image 3 and an in-reflecting-surface aerial image 4 using light emitted by an optical system 7. The on-reflecting-surface aerial image 3 is an aerial image displayed on a specular reflecting surface 2. The in-reflecting-surface aerial image 4 is an aerial image displayed as a reflected image within the specular reflecting surface 2. A user at a specified viewpoint position views the on-reflecting-surface aerial image 3 and the in-reflecting-surface aerial image 4. In this embodiment, the space above the specular reflecting surface 2, i.e., the space in which the user views objects in the air, is referred to as "on the reflecting surface," and the space below the specular reflecting surface 2, i.e., the space in which the user views objects within the specular reflecting surface 2, is referred to as "inside the reflecting surface."

[0012] As shown in FIG. 1, the optical system 7 includes, for example, a display 71, a beam splitter 72, a quarter-wave plate 73, a retroreflector 74, and the like.

[0013] The display 71 is a display capable of displaying an image for the aerial image 3 and an image for the in-plane aerial image 4 .

[0014] The beam splitter 72 is, for example, a polarizing beam splitter, which transmits light of the same polarization state and reflects light of the same polarization state.

[0015] The quarter-wave plate 73 and the retroreflector 74 are attached together and disposed above the display 71 .

[0016] In FIG. 1 , the optical path indicated by the solid line from the optical system 7 is for displaying the aerial image 3 on the reflecting surface, and the optical path indicated by the dotted line is for displaying the aerial image 4 within the reflecting surface. As shown in FIG. 1 , light emitted from the bottom of the display 71 is reflected by the beam splitter 72, retroreflected by the retroreflector 74, transmitted through the beam splitter, and specularly reflected by the specular reflecting surface 2, forming an aerial image on the reflecting surface. The user then visually recognizes this light as the aerial image 3 on the reflecting surface. Meanwhile, light emitted from the top of the display 71 is reflected by the beam splitter 72, retroreflected by the retroreflector 74, transmitted through the beam splitter 72, and formed at the same position as the aerial image 3 on the reflecting surface. After forming the image, the light is specularly reflected by the specular reflecting surface 2 and then travels toward the specified viewpoint position. As a result, the user visually recognizes the aerial image within the reflecting surface.

[0017] The light traveling directly from the display 71 toward the user may be absorbed by an absorbing polarizer attached to the beam splitter 72. This prevents the user from directly viewing the image displayed on the display 71 when viewing the aerial image.

[0018] 1 is merely an example, and the present embodiment is not limited to this configuration. For example, the optical system 7 may have any configuration as long as it is an optical system 7 that can display the on-reflecting surface aerial image 3 and the in-reflecting surface aerial image 4.

[0019] 2 is a block diagram showing an example of the hardware configuration and software configuration of an aerial image display device 1 according to a first embodiment. First, the hardware configuration of the aerial image display device 1 will be described.

[0020] 2, the aerial image display device 1 includes a control unit 11, a program storage unit 12, a data storage unit 13, a communication interface 14, and an input / output interface 15. The control unit 11, the program storage unit 12, the data storage unit 13, the communication interface 14, and the input / output interface 15 are communicatively connected to one another via a bus. Furthermore, the communication interface 14 may be communicatively connected to an external device via a network. Furthermore, the input / output interface 15 is communicatively connected to the input device 5, the output device 6, the optical system 7, the reference light source 8, and the luminance meter 9.

[0021] The control unit 11 controls the aerial image display device 1. The control unit 11 includes a hardware processor such as a central processing unit (CPU). For example, the control unit 11 may be an integrated circuit capable of executing various programs.

[0022] The program storage unit 12 may use, as a storage medium, a combination of nonvolatile memory that can be written to and read from at any time, such as an erasable programmable read-only memory (EPROM), a hard disk drive (HDD), or a solid state drive (SSD), and a nonvolatile memory such as a read-only memory (ROM). The program storage unit 12 stores programs necessary for executing various processes. That is, the control unit 11 can realize various controls and operations by reading and executing the programs stored in the program storage unit 12.

[0023] The data storage unit 13 is a storage that uses a combination of nonvolatile memory such as a HDD or memory card that can be written to and read from at any time, and volatile memory such as a RAM (Random Access Memory), as a storage medium. The data storage unit 13 is used to store data acquired and generated in the process of the control unit 11 executing programs and performing various processes.

[0024] The communication interface 14 includes one or more wired or wireless communication modules. For example, the communication interface 14 includes a communication module that establishes a wired or wireless connection with an external device via a network. The communication interface 14 may also include a wireless communication module that establishes a wireless connection with an external device such as a Wi-Fi access point or base station. Furthermore, the communication interface 14 may also include a wireless communication module that establishes a wireless connection with an external device using short-range wireless technology. In other words, the communication interface 14 may be any general communication interface that can communicate with an external device and send and receive various information under the control of the control unit 11.

[0025] The input / output interface 15 is connected to the input device 5, the output device 6, the optical system 7, the reference light source 8, the luminance meter 9, etc. The input / output interface 15 is an interface that enables transmission and reception of information between the input device 5, the output device 6, the optical system 7, the reference light source 8, and the luminance meter 9. The input / output interface 15 may be integrated with the communication interface 14. For example, the aerial image display device 1 and at least one of the input device 5, the output device 6, the optical system 7, the reference light source 8, and the luminance meter 9 may be wirelessly connected using short-range wireless technology or the like, and information may be transmitted and received using the short-range wireless technology.

[0026] The input device 5 may include, for example, a keyboard, a pointing device, or the like for the user to input various information to the aerial image display device 1. The input device 5 may also include a reader for reading data to be stored in the program storage unit 12 or the data storage unit 13 from a memory medium such as a USB memory, or a disk device for reading such data from a disk medium.

[0027] The output device 6 includes a display or the like that displays the results of calculations performed by the control unit 11. The output device 6 also includes a printer or the like that prints out information displayed on the display.

[0028] The optical system 7 includes at least a light source 71. Furthermore, the optical system 7 may include each of the devices described above. The light source 71 is a display having two ranges capable of displaying different brightness levels. For example, the light source 71 is a display having a range (first range) for displaying an aerial image on a reflective surface and a range (second range) for displaying an aerial image within the reflective surface.

[0029] Fig. 3 is a diagram showing an example of a light source 71 according to the first embodiment. As shown in Fig. 3, the light source 71 has a first range 711 and a second range 712. The first range 711 is a range in which the maximum luminance of the display image is x and an aerial image is displayed on the reflective surface. The second range 712 is a range in which the maximum luminance of the display image is y and an aerial image is displayed on the reflective surface.

[0030] The reference light source 8 is a light source used when calculating the reflectance of the specular reflecting surface 2. The reference light source 8 may be the light source 71 included in the optical system 7.

[0031] The luminance meter 9 is a device for measuring luminance that is placed at the specified viewpoint position shown in FIG. 1 . The luminance meter 9 measures the luminance at the specified viewpoint position and outputs the measurement result to the control unit 11. Note that the luminance meter 9 may be any device capable of measuring luminance, and therefore a detailed description thereof will be omitted here. Note that the reference light source 8 and the luminance meter 9 are used in advance to measure the reflectance of the specular reflective surface 2. For example, a service provider who displays the aerial image may use them in advance to measure the reflectance. Therefore, it goes without saying that the aerial image display device 1 does not need to include the reference light source 8 and the luminance meter 9 when displaying the on-reflecting surface aerial image 3 and the in-reflecting surface aerial image 4. For example, when a user uses the aerial image display device 1, the aerial image display device 1 does not need to include the reference light source 8 and the luminance meter 9.

[0032] (Operation) Next, an example of information processing operation performed by the aerial image display device 1 according to the embodiment configured as described above will be described. (Reflectance Measurement Operation) First, as described above, the reflectance of the specular reflecting surface 2 varies depending on the material, reflection angle, etc. These reflectances α must be stored in the specular reflecting surface database 131. Therefore, first, the operation of the reflectance measurement method will be described.

[0033] FIG. 4 is a diagram showing an example of the reference light source 8 and luminance meter 9 when measuring the luminance measurement value R according to the first embodiment. As shown in FIG. 4, the luminance measurement value R is obtained using the reference light source 8 and luminance meter 9. As shown in FIG. 4, distance C indicates the horizontal distance from the aerial image display position to the luminance meter 9. Height H indicates the vertical distance between the specular reflecting surface 2 and the luminance meter 9. Aerial image height D indicates the height of the aerial image. Furthermore, angle θ is the angle formed by the aerial image and a line connecting the specified viewpoint position and the center of the aerial image (i.e., D / 2). Here, the reflectance measurement unit 111 calculates the angle θ using the following equation:

[0034]

[0035] 4, the luminance meter 9 measures the luminance measurement value R, which is the measurement value of the luminance of the reference light source 8 itself. The luminance meter 9 outputs the measured luminance measurement value R to the reflectance measurement unit 111.

[0036] 5 is a diagram showing an example of the reference light source 8 and the luminance meter 9 when measuring the reflected image measurement value RR according to the first embodiment. As shown in FIG. 5, the reflected image measurement value RR is measured using the reference light source 8, the luminance meter 9, and the specular reflecting surface 2.

[0037] As shown in FIG. 5, the reference light source 8 is placed on the specular reflection surface 2, and the luminance meter 9 is placed at a position C+D tan θ horizontally from the reference light source 8 and H vertically from the specular reflection surface 2.

[0038] The luminance meter 9 measures the reflected image 81 of the reference light source displayed on the specular reflecting surface 2 in the arrangement shown in Figure 5, thereby measuring the reflected image measurement value RR. The luminance meter 9 outputs the measured reflecting surface measurement value RR to the reflectance measurement unit 111.

[0039] The reflectance measurement unit 111 calculates the reflectance α using the following equation 2. Then, the calculated reflectance α is stored in the specular reflection surface database 131.

[0040] α=RR / R (Formula 2)

[0041] (Aerial Image Display Operation) Next, the display operation of the on-reflecting surface aerial image 3 and the in-reflecting surface aerial image 4 after measuring the reflectance α will be described. Fig. 6 is a flowchart showing an example of the aerial image display operation by the aerial image display device 1 according to the first embodiment. The operation of this flowchart is realized by the control unit 11 of the aerial image display device 1 reading and executing a program stored in the program storage unit 12.

[0042] For example, this flowchart starts when the user instructs the aerial image display device 1 to display an arbitrary aerial image.

[0043] In step ST101, the reflected image luminance calculation unit 112 calculates the luminance for the aerial image to be displayed on the light source 71. For example, the upper limit of the luminance of the image in the first range 711 is set to luminance x, and the upper limit of the luminance of the image in the second range 712 is set to luminance y. The reflected image luminance calculation unit 112 then calculates the luminance y using the reflectance α stored in the specular reflection surface database 131, using the following equation 3. Here, the luminance x is set to the maximum luminance M of the light source 71. Note that the luminance x may be any value equal to or less than the maximum luminance M of the light source 71. The luminance x may be determined by a user instruction, or the reflected image luminance calculation unit 111 may use a luminance x that is predetermined.

[0044] y=x×α (Equation 3) The reflected image luminance calculation unit 112 outputs the luminance x, the luminance y, and the reflectance α to the image generation unit 113 .

[0045] In step ST102, the image generation unit 113 generates images for the aerial image and the reflected aerial image. The image generation unit 113 generates images for the aerial image and the reflected aerial image to be displayed on the light source 71 using the luminance x and y of the first range 711 and the second range 712. The image generation unit 113 outputs the generated images for the aerial image and the reflected aerial image to the aerial image display control unit 114.

[0046] In step ST103, the aerial image display control unit 114 controls the display of aerial images on and within the reflecting surface using images for the aerial image and the reflected aerial image. The aerial image display control unit 114 displays the image for the aerial image in the first range 711 of the light source 71 and the image for the reflected aerial image in the second range 712, thereby displaying the aerial image and the reflected aerial image on and within the reflecting surface, respectively.

[0047] 7 is a diagram showing a display example of the aerial image on the reflecting surface 3 and the aerial image in the reflecting surface 4 according to the first embodiment. As shown in Fig. 7, a user at a specified viewpoint position can view the aerial image on the reflecting surface 3 and the aerial image in the reflecting surface 4. Note that for simplicity, the internal configuration of the optical system 7 is omitted from Fig. 7.

[0048] (Effects of the First Embodiment) According to the first embodiment, the reflectance α of each specular reflecting surface 2 is measured and stored in the specular reflecting surface database 131. The aerial image display device 1 can then display the on-reflecting surface aerial image 3 and the in-reflecting surface aerial image 4 in accordance with the reflectance α of each specular reflecting surface 2.

[0049] [Modification of First Embodiment] In a modification of the first embodiment, the reflectance α of the specular reflective surface 2 at the angle θ is calculated by the following calculation without using the measured values ​​from the reference light source 8 and the luminance meter 9.

[0050] (Configuration) The modified example of the first embodiment has the same configuration as the aerial image display device 1 described with reference to Fig. 2, except that it does not use the reference light source 8 and the luminance meter 9. Therefore, a duplicated description will be omitted here.

[0051] (Operation) (Reflectance Measurement Operation) In a modification of the first embodiment, the reflectance α is calculated using Fresnel's formula. For example, the ratio of the amplitude of the electric field of the reflected light and the transmitted light when light is incident on the specular reflective surface 2 of a medium with a different refractive index is calculated. The reflection angle θ is calculated from the following equation 4.

[0052]

[0053] When the polarization of the light incident on the specular reflecting surface 2 is s-polarized, Fresnel's formula is expressed as the following Equation 5. The refraction angle of the specular reflecting surface 2 is written as γ.

[0054]

[0055] Here, the refraction angle γ of the specular reflecting surface 2 is calculated from the following equation 6, which is Snell's law. Note that the refractive index of the specular reflecting surface 2 is n, which is a known value. Here, 1.0003 is the refractive index of air.

[0056]

[0057] For example, the reflectance measurement unit 111 calculates the reflectance α using the method described above, and stores the calculated reflectance α in the specular reflecting surface database 131 .

[0058] (Aerial Image Display Operation) The aerial image display operation is the same as in the first embodiment, and therefore a duplicated description will be omitted here.

[0059] (Effects of the Modification of the First Embodiment) According to the modification of the first embodiment, the aerial image display device 1 calculates the reflectance α by the calculation described above without measuring the reflectance α for each specular reflecting surface 2, and stores the calculated reflectance α in the specular reflecting surface database 131. The aerial image display device 1 can then display the on-reflecting surface aerial image 3 and the in-reflecting surface aerial image 4 in accordance with the reflectance α for each specular reflecting surface 2.

[0060] Second Embodiment (Configuration) The second embodiment has the same configuration as the aerial image display device 1 described with reference to Fig. 2. Therefore, a duplicated description will be omitted here.

[0061] (Operation) (Reflectance measurement operation) In the second embodiment, the brightness of the aerial image 3 on the reflecting surface and the aerial image 4 within the reflecting surface, the brightness of the reference light source 8 on the specular reflecting surface 2, and the brightness of the reference light source 8 that has passed through the specular reflecting surface 2 are used.

[0062] FIG. 8 is a diagram showing an example of measuring the luminance of the aerial image 3 on the reflecting surface and the aerial image 4 in the reflecting surface according to the second embodiment. As shown in FIG. 8 , an optical system 7, a luminance meter 9, and a specular reflecting surface 2 are used to measure the luminance. First, as described in the first embodiment, the optical system 7 is used to display the aerial image 3 on the reflecting surface and the aerial image 4 in the reflecting surface. Then, as in FIG. 4 , the luminance meter 9 is placed at a horizontal distance C from the aerial image and a vertical height H from the specular reflecting surface 2. The luminance meter 9 is then used to measure the luminance of the aerial image 3 on the reflecting surface and the aerial image 4 in the reflecting surface. For example, if the aerial image height of the aerial image 3 on the reflecting surface and the aerial image 4 in the reflecting surface is D, the luminance meter 9 measures the luminance at the center (i.e., D / 2) of the aerial image 3 on the reflecting surface and the aerial image 4 in the reflecting surface. For example, let A be the measured luminance of the aerial image 3 on the reflecting surface, and AR be the measured luminance of the aerial image 4 in the reflecting surface. The reflectance measurement unit 111 acquires the measurement values ​​A and AR measured by the luminance meter 9 and stores them in the specular reflection surface database 131 .

[0063] FIG. 9 is a diagram showing an example of measuring the luminance of a reference light source 8 on a specular reflecting surface 2 and the luminance of the reference light source 8 passing through the specular reflecting surface 2 according to the second embodiment. As shown in FIG. 9 , the luminance measurement uses a reference light source 8, a luminance meter 9, and a specular reflecting surface 2. The reference light source 8 is placed on the specular reflecting surface 2, and the luminance meter 9 is placed at a horizontal distance C from the reference light source 8 and a vertical height H from the specular reflecting surface. The luminance meter 9 is then used to measure the luminance of the reference light source 8 and the luminance of a reflected image 81 of the reference light source reflected on the specular reflecting surface 2. For example, if the height of the reference light source 8 is D, the luminance meter 9 measures the luminance of the reference light source 8 on the reflecting surface and at the center (i.e., D / 2) of the reflected image of the reference light source 8 within the reflecting surface. For example, the luminance of the reference light source 8 is represented by R, and the luminance of the reflected image of the reference light source 8 is represented by RR. The reflectance measurement unit 111 acquires the measured values ​​R and RR measured by the luminance meter 9 and stores them in the specular reflecting surface database 131.

[0064] (Aerial Image Display Operation) Next, a display operation of on-reflecting surface aerial image 3 and in-reflecting surface aerial image 4 will be described using the measured values.

[0065] 10 is a flowchart showing an example of an aerial image display operation by the aerial image display device 1 according to the second embodiment. The operation of this flowchart is realized by the control unit 11 of the aerial image display device 1 reading and executing a program stored in the program storage unit 12.

[0066] For example, this flowchart starts when the user instructs the aerial image display device 1 to display an arbitrary aerial image.

[0067] In step ST201, the reflected image luminance calculation unit 112 calculates the luminance for the aerial image to be displayed on the light source 71. For example, the reflected image luminance calculation unit 112 adjusts the luminance of the light source 71 so that the luminance ratio between the on-reflecting-surface aerial image 3 and the in-reflecting-surface aerial image 4 becomes R:RR.

[0068] For example, the upper limit of the luminance of the image in the first range 711 is set to luminance x, and the upper limit of the luminance of the image in the second range 712 is set to luminance y. Then, the reflected image luminance calculation unit 112 calculates the luminance x and the luminance y using the measurement values ​​(R, RR, A, AR) stored in the specular reflection surface database 131. For example, the reflected image luminance calculation unit 112 performs the calculation using the following equation 7. Here, the luminance x is set to the maximum luminance M of the light source 71. Note that the luminance x may be any value equal to or less than the maximum luminance M of the light source 71. The luminance x may be determined by a user instruction, or the reflected image luminance calculation unit 111 may use a luminance x that is determined in advance.

[0069]

[0070] In step ST202, the image generation unit 113 generates images for the aerial image and the reflected aerial image. The image generation unit 113 generates images for the aerial image and the reflected aerial image to be displayed on the light source 71 using the luminance x and y of the first range 711 and the second range 712. The image generation unit 113 outputs the generated images for the aerial image and the reflected aerial image to the aerial image display control unit 114.

[0071] In step ST203, the aerial image display control unit 114 performs control to display aerial images on the specular reflecting surface 2 using images for the aerial image and the reflected aerial image. The aerial image display control unit 114 displays the image for the aerial image in the first range 711 of the light source 71 and the image for the reflected aerial image in the second range 712, thereby displaying the on-reflecting surface aerial image 3 and the in-reflecting surface aerial image 4 on the specular reflecting surface 2.

[0072] 11 is a diagram showing a display example of the aerial image on the reflecting surface 3 and the aerial image in the reflecting surface 4 according to the second embodiment. As shown in FIG. 11 , a user at a specified viewpoint position can view the aerial image on the reflecting surface 3 and the aerial image in the reflecting surface 4. In the second embodiment, the luminance displayed in the first range 711 and the second range 712 is adjusted to the values ​​calculated in step ST201, so that the luminance ratio between the aerial image on the reflecting surface 3 and the aerial image in the reflecting surface 4 becomes R:RR.

[0073] (Effects of the Second Embodiment) According to the second embodiment, measurement results of the luminance of the aerial image 3 on the reflecting surface and the aerial image 4 within the reflecting surface, the luminance of the reference light source 8 on the specular reflecting surface 2, and the luminance of the reference light source 8 that has passed through the specular reflecting surface 2 are stored in the specular reflecting surface database 131. Then, based on the measurement results stored in the specular reflecting surface database 131, the aerial image display device 1 can reproduce the aerial images 3 on the reflecting surface and the aerial images 4 within the reflecting surface displayed on various specular reflecting surfaces 2 in accordance with the reflectance of each specular reflecting surface 2 and taking into account the directionality of the light.

[0074] [Other Embodiments] The present invention is not limited to the above-described embodiments. The flow of each process described above is not limited to the described procedure, and the order of some steps may be changed, or some steps may be performed simultaneously in parallel. Furthermore, the series of processes described above do not need to be performed consecutively, and each step may be performed at any timing.

[0075] Furthermore, the techniques described in the above embodiments can be stored as a program (software means) that can be executed by a computer on a storage medium such as a magnetic disk (e.g., a floppy disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only execution programs but also tables and data structures) that the computer executes. The computer that realizes this device loads the program stored on the storage medium and, in some cases, configures the software means using the configuration program, and executes the above-mentioned processing by controlling the operation of this software means. Note that the term "storage medium" as used herein is not limited to storage media for distribution, but also includes storage media such as magnetic disks and semiconductor memories installed inside the computer or in devices connected via a network.

[0076] In short, this invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in combination as appropriate as possible, and in such cases, the combined effects can be obtained. Furthermore, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining the disclosed multiple constituent elements.

[0077] DESCRIPTION OF SYMBOLS 100...Aerial image display system 1...Aerial image display device 11...Control unit 111...Reflectance measurement unit 112...Reflected image brightness calculation unit 113...Image generation unit 114...Aerial image display control unit 12...Program memory unit 13...Data memory unit 131...Specular reflection surface database 14...Communication interface 15...Input / output interface 2...Specular reflection surface 3...Aerial image on reflection surface 4...Aerial image in reflection surface 5...Input device 6...Output device 7...Optical system 71...Light source 711...First range 712...Second range 72...Beam splitter 73...1 / 4λ wavelength plate 74...Retroreflective material 8...Reference light source 81...Reflected image of reference light source 9...Luminance meter

Claims

1. An aerial image display device comprising: a memory unit that stores the reflectance of a specular reflecting surface; a reflected image brightness calculation unit that determines the brightness of an aerial image on the specular reflecting surface to be displayed on the specular reflecting surface and the brightness for displaying an aerial image within the specular reflecting surface to be displayed within the specular reflecting surface based on the reflectance; and a display control unit that controls an optical system to display the aerial image on the specular reflecting surface and the aerial image within the specular reflecting surface using the determined brightness.

2. The aerial image display device according to claim 1, wherein the optical system includes a light source, the light source includes a first range for displaying an aerial image on the specular reflecting surface and a second range for displaying an aerial image within the specular reflecting surface, and the reflected image brightness calculation unit determines the brightness of the first range and the second range.

3. An aerial image display method executed by a processor of an aerial image display device, comprising: storing the reflectance of a specular reflecting surface; determining, based on the reflectance, a brightness for displaying an aerial image on the specular reflecting surface to be displayed on the specular reflecting surface and a brightness for displaying an aerial image within the specular reflecting surface to be displayed within the specular reflecting surface; and controlling an optical system to display the aerial image on the specular reflecting surface and the aerial image within the specular reflecting surface using the determined brightness.

4. An aerial image display program comprising instructions to be executed by a processor of an aerial image display device, the instructions comprising: storing the reflectance of a specular reflecting surface; determining, based on the reflectance, a brightness for displaying an aerial image on the specular reflecting surface and an aerial image within the specular reflecting surface, and using the determined brightness to control an optical system to display the aerial image on the specular reflecting surface and the aerial image within the specular reflecting surface.

Images