Information processing device, method, and program

Abstract

According to one embodiment, an information processing device includes an aerial image display unit that reflects light to display an aerial image that is formed in the air, a video display unit that displays video of a surface related to the aerial image that is a reflection surface for the light reflected by the aerial image display unit, and a control unit that, when an operation that changes the details of the video displayed by the video display unit is performed, controls the details of the display of the aerial image by the aerial image display unit on the basis of the details of the operation.

Description

Information processing device, method and program

[0001] FIELD Embodiments of the present invention relate to an information processing device, method, and program.

[0002] One method of interaction between a user and a mid-air image, which is a video image formed and displayed as a real image in the air using optical elements, is to sense the user's hand that is extended directly toward the mid-air image. For example, "MiTAi" disclosed in Non-Patent Document 1 implements an interaction in which a tracking sensor is used to detect the position of the user's fingertip, and a dragon, which is a mid-air image, follows this position with its eyes.

[0003] Motohiro Makiguchi, Ayaka Sano, Takahiro Matsumoto, Hiroshi Chigira, Takayoshi Mochiduki, "Implementation of Interactive Mirror-Transcending Aerial Imaging System", Proceedings of the 2023 ACM Symposium on Spatial User Interaction

[0004] On the other hand, a relatively large number of users find it difficult to understand the display position of the aerial image, and since there is no tactile feedback even when the user touches the aerial image, it takes a long time for the user to get used to operating the aerial image.

[0005] The present invention has been made in light of the above-mentioned circumstances, and its object is to provide an information processing device, method, and program that are capable of appropriately displaying aerial images in accordance with operations.

[0006] An information processing device according to one aspect of the present invention comprises an aerial image display unit that displays an aerial image formed in the air by reflecting light, an image display unit that displays an image of a surface related to the aerial image, which forms the reflective surface of the light reflected by the aerial image display unit, and a control unit that, when an operation is performed to change the content of the image displayed by the image display unit, controls the display content of the aerial image by the aerial image display unit based on the content of the operation.

[0007] An information processing method according to one aspect of the present invention is a method performed by an information processing device, and includes the steps of: displaying an aerial image formed in the air by reflecting light using an aerial image display unit of the information processing device; displaying an image of a surface related to the aerial image, which forms a reflective surface for light reflected by the aerial image display unit, using a video display unit of the information processing device; and, when an operation is performed to change the content of the video displayed by the video display unit, controlling the display content of the aerial image by the aerial image display unit based on the content of the operation using a control unit of the information processing device.

[0008] According to the present invention, it is possible to appropriately display an aerial image in accordance with an operation.

[0009] FIG. 1 is a diagram illustrating an example application of an information processing device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating an example of an optical system for forming an aerial image. FIG. 3 is a diagram illustrating an example of a processing flow by each unit in the information processing device. FIG. 4 is a flowchart illustrating an example of a processing operation procedure by the information processing device according to an embodiment of the present invention. FIG. 5 is a diagram illustrating an example application of vibration presentation by the information processing device according to an embodiment of the present invention. FIG. 6 is a flowchart illustrating an example of a processing procedure for vibration presentation by the information processing device according to an embodiment of the present invention. FIG. 7 is a diagram illustrating an example application of warmth presentation by the information processing device according to an embodiment of the present invention. FIG. 8 is a flowchart illustrating an example of a processing procedure for warmth presentation by the information processing device according to an embodiment of the present invention. FIG. 9 is a diagram illustrating an example application of indirect manipulation using a shadow of an aerial image by the information processing device according to an embodiment of the present invention. FIG. 10 is a flowchart illustrating an example of a processing procedure for joint manipulation using a shadow of an aerial image by the information processing device according to an embodiment of the present invention. FIG. 11 is a block diagram illustrating an example of the hardware configuration of an information processing device according to an embodiment of the present invention.

[0010] An embodiment of the present invention will be described below. In this embodiment, a configuration will be described in which an aerial image, which is an image displayed as a real image formed using optical elements, is indirectly manipulated by manipulating the surrounding environment of the aerial image using a touch panel. FIG. 1 is a diagram showing an application example of an information processing device according to an embodiment of the present invention. As shown in FIG. 1, an information processing device 100, which is an image processing device according to an embodiment of the present invention, includes a virtual environment control unit 10, a touch panel 20, and an aerial image display unit 30. The touch panel 20 may be a combination of a device having a touch function and a display device.

[0011] 1 , virtual environment control unit 10 has a bottom display control unit 11 and an aerial image display control unit 12. Touch panel 20 is formed by combining a display device such as a liquid crystal panel with a pointing device such as a touchpad, and has a touch detection unit 21 and a bottom display unit 22.

[0012] In this embodiment, the information processing device 100 detects a user input on the touch panel 20, performs processing in response to the input, and then outputs the processing results to the bottom display unit 22 and the aerial image display unit 30. In this embodiment, the touch panel 20 is installed so that its display screen forms a surface related to the aerial image and serves as a bottom surface relative to the aerial image, and the touch detection unit 21 detects a user's touch operation on the display screen. The surface related to the aerial image serves as a reflective surface (described later) for forming the aerial image and a surface displaying the surrounding environment relative to the aerial image, such as a bottom surface, rear surface, or side surface relative to the position where the aerial image video (sometimes referred to as aerial image video or simply aerial image) is displayed. The bottom display unit 22 displays the bottom video (sometimes referred to as a bottom video) at a position corresponding to the bottom surface relative to the position where the aerial image video is displayed. The touch panel 20 may also be installed by moving it to, for example, a wall surface, as long as it serves as a reflective surface and displays the surrounding environment relative to the aerial image. In this case, the bottom display unit 22 is used as an image display unit corresponding to the surface on which the touch panel 20 is installed. Also, a plurality of touch panels 20 may be provided.

[0013] The aerial image display unit 30 displays an aerial image formed by using light reflection to form an aerial image. The virtual environment control unit 10 controls the virtual environment including the bottom surface and the aerial image projected on the bottom surface.

[0014] FIG. 2 illustrates an example of an optical system for forming an aerial image. As illustrated in FIG. 2 , the aerial image display unit 30 includes an aerial image light source display 31, a beam splitter 32, and a retroreflector 33. The beam splitter 32 is an optical element that transmits a portion of incident light and reflects a portion of it. In this embodiment, a reflective polarizing film is attached to the touch panel 20, forming a reflective surface as part of the optical system for forming an aerial image. The optical system shown in FIG. 2 forms an aerial image by retroreflection, where light from the light source display 31 is formed at a plane-symmetrical position with respect to the beam splitter 32 and at a plane-symmetrical position with respect to the surface of the touch panel 20. Furthermore, polarization characteristics are utilized to efficiently utilize the brightness of the light from the light source display 31 while eliminating stray light that travels directly from the light source display 31 toward the user.

[0015] 2, an absorbing polarizing film 31a is attached to the light source display 31, a reflective polarizing film 32a is attached to the beam splitter 32, and a quarter-wave plate 33a, which is a retardation film, is attached to the retroreflective material 33. However, in the example shown in Fig. 2, the polarization direction of the light emitted from the light source display 31 and the reflective polarization direction of the beam splitter 32 are made to coincide with each other, and these directions are made perpendicular to the reflective polarization direction of the touch panel 20.

[0016] Light emitted from the light source display 31, for example, p-polarized light, is reflected by the reflective polarizing film 32a of the beam splitter 32 and retroreflected by the phase difference film of the retroreflector 33, whereby it is converted into s-polarized light. This light passes through the reflective polarizing film 32a of the beam splitter 32 and is reflected by the reflective polarizing film of the touch panel 20, thereby forming an aerial image (reference symbol a in FIG. 2 ). Reference symbol b in FIG. 2 indicates the reflection and transmission points of the light emitted from the light source display 31, and reference symbol c in FIG. 2 indicates the retroreflection points of the light emitted from the light source display 31.

[0017] Light traveling directly from light source display 31 toward the user does not pass through reflective polarizing film 32a of beam splitter 32, so the user does not perceive the direct light from light source display 31 as stray light. Assuming that light source display 31 emits light with no bias in polarization direction, reflective polarizing film 32a transmits and reflects light in all directions without loss, and retroreflector 33 retroreflects incident light without loss, the theoretical brightness efficiency of the optical system shown in Figure 2 is such that 50% of the light emitted from light source display 31 can be used for the aerial image.

[0018] 2, mirrors 34 are arranged on the left and right sides of the housing of the aerial image display unit 30, similar to a known technique of using mirrors in an aerial image optical system using retrotransmission. This allows the field of view to be expanded. For example, light that would otherwise be incident on a wall surface can be used to form an aerial image, thereby increasing the horizontal field of view.

[0019] 3 is a diagram showing an example of the flow of processing by each unit of the information processing device. The touch detection unit 21 of the touch panel 20 performs the following input, processing, and output: Input: A touch operation by a user, for example, a touch operation by the user's hand or a pen-like object held by the user, a so-called touch pen, is input. Processing: The coordinates of the point where the touch operation was performed (sometimes referred to as touch coordinates) or, for example, the transition of the coordinates of points where consecutive touch operations were performed within a certain period of time, are recorded as the content of the touch operation. Output: Information on the processing result is sent to the virtual environment control unit 10.

[0020] The virtual environment control unit 10 performs the following input, processing, and output. Input: Receives information on the detection result of the touch operation from the touch detection unit 21. Processing: Controls the display of the bottom image and the aerial image according to the coordinates of the location where the touch operation was performed or the transition of those coordinates, as indicated by the detection result, and cuts out the bottom image and the aerial image. Output: Sends the bottom image to the bottom display unit 22 and the aerial image to the aerial image display unit 30.

[0021] The bottom display unit 22 performs the following input and output: Input: Receives a bottom image from the virtual environment control unit 10. Output: Outputs a bottom image.

[0022] The aerial image display unit 30 performs the following input and output: Input: Receives an image for an aerial image from the virtual environment control unit 10. Output: Outputs an image for an aerial image.

[0023] The information processing device 100 according to this embodiment processes the detected touch coordinates, the trajectory of those coordinates, and the detection period in the virtual environment control unit 10, generates an image for the bottom surface and an image for the aerial image, and outputs them to each display unit.

[0024] In this embodiment, tactile feedback can be provided relatively easily in response to touch operations. For example, because the user is actually touching the display screen of the touch panel, interaction with the aerial image is easy. Furthermore, the user is less likely to be confused and can easily become accustomed to the interaction-related operations. Next, indirect operation of the aerial image via the touch panel, which is the basic configuration according to this embodiment shown in FIG. 1 , will be described. FIG. 4 is a flowchart showing an example of the processing procedure of the information processing device according to one embodiment of the present invention. First, when the touch detection unit 21 detects a touch operation on the operation surface of the touch panel 20 by the user (S1), the bottom display control unit 11 of the virtual environment control unit 10 controls the content of the bottom image displayed by the bottom display unit 22 according to the content of the touch operation, such as the position information of the touch operation (S2).

[0025] Furthermore, the aerial image display control unit 12 of the virtual environment control unit 10 controls the content of the video for the aerial image displayed by the aerial image display unit 30 in accordance with the content of the position information of the touch operation and the like (S3).

[0026] In this configuration, the aerial image is manipulated by touching the bottom image projected at the feet of the aerial image. For example, by displaying a bottom image of a stage on the bottom display unit 22 of the touch panel 20 at the feet of the aerial image displayed by the aerial image display unit 30, the aerial image appears to be standing on the stage. Furthermore, by moving the bottom image of the stage displayed by the bottom display unit 22 on the display screen of the touch panel 20 through a touch operation, the position of the aerial image displayed by the aerial image display unit 30 is moved in conjunction with this movement. Furthermore, by rotating the bottom image of the stage displayed by the bottom display unit 22 on the display screen of the touch panel 20 through a touch operation, the aerial image displayed by the aerial image display unit 30 is rotated in conjunction with this rotation. Furthermore, by poking the bottom image of the stage displayed by the bottom display unit 22 on the display screen of the touch panel 20 through a touch operation, the aerial image displayed by the aerial image display unit 30 is displayed as shaking in conjunction with this operation.

[0027] By using the configuration shown in FIG. 1, a user can interact with a mid-air image using familiar touch operations, without having to perform gestures such as touching the mid-air image directly with their hands.

[0028] (Application Example of Vibration Presentation) Fig. 5 is a diagram showing an application example of vibration presentation by an information processing device according to an embodiment of the present invention. The information processing device 100a shown in Fig. 5 includes a virtual environment control unit 10, a touch panel 20, and an aerial image display unit 30. The virtual environment control unit 10 has a bottom display control unit 11, an aerial image display control unit 12, and a bottom vibration control unit 13. The touch panel 20 has a touch detection unit 21, a bottom display unit 22, and a bottom vibration unit 23.

[0029] With the conventional method of sensing a hand in the air, it is difficult to provide a wide range of types of tactile feedback. Therefore, in the application example shown in Figure 5, the touch panel 20 is vibrated using, for example, a vibrator function, so that vibration feedback is provided to the user in addition to the feeling of contact with the touch panel by the user.

[0030] The bottom vibration control unit 13 of the virtual environment control unit 10 performs the following input, processing, and output to present vibration. Input: Receives information such as the coordinates and transition of the touch position, which is the detection result of the touch operation, from the touch detection unit 21. Processing: Determines the coordinates of the position on the touch panel 20 where vibration should occur based on the content of the touch operation. Output: Vibrates the position of the determined coordinates on the touch panel 20 by the bottom vibration unit 23.

[0031] 6 is a flowchart showing an example of a processing procedure for vibration presentation by an information processing device according to an embodiment of the present invention. First, when a touch operation by a user on the operation surface of the touch panel 20 is detected by the touch detection unit 21 (S11), the bottom display control unit 11 of the virtual environment control unit 10 controls the content of the bottom image displayed by the bottom display unit 22 in accordance with the content of the touch operation (S12).

[0032] Furthermore, the bottom vibration control unit 13 of the virtual environment control unit 10 controls the vibration content of the bottom vibration unit 23 of the touch panel 20 according to the position information of the touch operation (S13).

[0033] Furthermore, the aerial image display control unit 12 of the virtual environment control unit 10 controls the content of the video for the aerial image displayed by the aerial image display unit 30 in accordance with the content of the touch operation (S14).

[0034] 5, vibrations in response to a touch operation that shakes the bottom image, which is represented as the ground, can be transmitted not only to the content of the aerial image but also to the user's fingers. For example, when the bottom image, which is represented as the ground, is shaken by a touch operation, the user can feel the vibrations caused by the ground moving, and the image can be controlled so that a character in the aerial image falls in response to the shaking.

[0035] Furthermore, for example, when the bottom image, which is likened to a turntable, is rotated by touch operation, the user can feel the vibrations caused by the movement of the ground, and in conjunction with the rotation, the image control can be performed to rotate the character, which is an aerial image, for example, 360 degrees.

[0036] In the example shown in Figure 5, vibration feedback can be implemented in response to touch operations, enabling seamless interaction. By implementing such feedback, users can experience the effect of easily understanding how they performed an operation, along with the tactile sensation of the touch operation.

[0037] (Application Example of Warm Sensation Presentation) Figure 7 is a diagram showing an application example of warmth presentation by an information processing device according to an embodiment of the present invention. The information processing device 100b shown in Figure 7 includes a virtual environment control unit 10, a touch panel 20, and an aerial image display unit 30. The virtual environment control unit 10 has a bottom surface display control unit 11, an aerial image display control unit 12, and a warmth presentation control unit 14. The touch panel 20 has a touch detection unit 21, a bottom surface display unit 22, and a warmth presentation unit 24.

[0038] With the conventional method of sensing a hand in the air, it is difficult to provide a wide range of feedback. Therefore, in the application example shown in Fig. 7, a temperature sensation is presented to the user in response to a touch operation on the touch panel 20, for example, by using the characteristics of a Peltier element built into the housing of the touch panel 20, so that the user is given feedback of the temperature sensation in addition to the sensation of contact with the touch panel.

[0039] The warmth sensation presentation control unit 14 of the virtual environment control unit 10 performs the following input, processing, and output in order to present a warmth sensation. Input: Receives information such as the coordinates and transition of the touch position, which is the detection result of the touch operation, from the touch detection unit 21. Processing: Determines the degree of temperature increase / decrease and the coordinates of the position on the touch panel 20 based on the content of the touch operation. Output: Causes the warmth sensation presentation unit 24 to raise / lower the temperature at the position of the determined coordinates on the touch panel 20.

[0040] 8 is a flowchart showing an example of a processing procedure for providing a thermal sensation by an information processing device according to an embodiment of the present invention. First, when the touch detection unit 21 detects a touch operation on the operation surface of the touch panel 20 by the user (S21), the bottom display control unit 11 of the virtual environment control unit 10 controls the content of the bottom image displayed by the bottom display unit 22 in accordance with the content of the touch operation (S22).

[0041] Furthermore, the warmth sensation presentation control unit 14 of the virtual environment control unit 10 controls the content of the warmth sensation presented by the warmth sensation presentation unit 24 of the touch panel 20 in accordance with the position information of the touch operation (S23). Furthermore, the aerial image display control unit 12 of the virtual environment control unit 10 controls the content of the aerial image video displayed by the aerial image display unit 30 in accordance with the content of the touch operation (S24).

[0042] In the example shown in FIG. 7, for example, the temperature characteristics of the ground displayed on the touch panel 20 as a bottom image can be directly transmitted to the user's finger or the like.

[0043] For example, when the ground as the bottom image is changed to ice by touch operation, the aerial image is displayed as if it is freezing, and in conjunction with the above change, a warm sensation presentation control is performed to make the user's fingers feel cold.

[0044] As another example, when the ground in the bottom image is changed to a desert by touch operation, the aerial image is controlled to look hot, and in conjunction with the above change, control is also performed to make the user's fingers feel the heat.

[0045] In the example shown in Figure 7, feedback by providing a thermal sensation can be implemented in response to touch operations, enabling interaction without discomfort. By implementing such feedback, the user can experience the effect of easily understanding how the operation was performed, along with the tactile sensation of the touch operation.

[0046] (Application Example of Indirect Operation Using Shadow of Aerial Image) Figure 9 is a diagram showing an application example of indirect operation using the shadow of an aerial image by an information processing device according to an embodiment of the present invention. The information processing device 100c shown in Figure 9 includes a virtual environment control unit 10, a touch panel 20, and an aerial image display unit 30. The virtual environment control unit 10 includes a bottom surface display control unit 11c and an aerial image display control unit 12. The touch panel 20 includes a touch detection unit 21 and a bottom surface display unit 22.

[0047] 10 is a flowchart showing an example of a processing procedure for joint manipulation using the shadow of an aerial image performed by an information processing device according to an embodiment of the present invention. First, before a user performs a touch operation on the operation surface of the touch panel 20, the bottom surface display control unit 11c of the virtual environment control unit 10 controls the content of the bottom surface image displayed by the bottom surface display unit 22, including the shadow of the aerial image displayed by the aerial image display unit 30 (S31).

[0048] When the touch detection unit 21 detects a touch operation by the user on the operation surface of the touch panel 20, including an action on the shadow of the aerial image displayed by the bottom display unit 22 (S32), the bottom display control unit 11c of the virtual environment control unit 10 controls the content of the bottom image displayed by the bottom display unit 22 in accordance with the content of the touch operation (S33). Furthermore, the aerial image display control unit 12 of the virtual environment control unit 10 controls the content of the aerial image image displayed by the aerial image display unit 30 in accordance with the content of the touch operation (S34).

[0049] In the application example shown in Figure 9, a shadow of the aerial image is displayed on the bottom display unit 22 of the touch panel 20 before the touch operation, and the user touches this shadow with their finger to indirectly move the aerial image.

[0050] As a first example, when a touch operation is performed to touch the shadow of the aerial image displayed on the bottom display unit 22 in a so-called pinning manner, image control is performed in conjunction with this operation to stop the aerial image from moving. As a second example, when a touch operation is performed to pull the shadow of the aerial image displayed on the bottom display unit 22, display control is performed in conjunction with this operation to pull the aerial image displayed by the aerial image display unit 30 in the direction of the pull, or to change the shape of the aerial image.

[0051] 9, the aerial image can be indirectly manipulated using the shadow of the aerial image originally displayed on the touch panel 20, and therefore the aerial image can be indirectly manipulated without adding an object such as a stage as a bottom image. Furthermore, displaying the shadow of the aerial image as a bottom image has the effect of making it easier for the user to have the illusion that the aerial image is three-dimensional.

[0052] Fig. 11 is a block diagram showing an example of the hardware configuration of an information processing device according to an embodiment of the present invention. In the example shown in Fig. 11, the information processing device 100 according to the embodiment is configured, for example, by a server computer or a personal computer, and has a hardware processor 111A such as a CPU (Central Processing Unit). A program memory 111B, a data memory 112, an input / output interface 113, and a communication interface 114 are connected to this hardware processor 111A via a bus 115. The same applies to the information processing devices 100a, 100b, and 100c described separately.

[0053] The communication interface 114 includes, for example, one or more wireless communication interface units, and enables transmission and reception of information to and from a communication network. As the wireless interface, for example, an interface that adopts a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0054] An input device 200 and an output device 300 attached to the information processing device 100 and used by a user or the like are connected to the input / output interface 113. The input device 200 and the output device 300 may include the touch panel 20 shown in FIG. 1. The input / output interface 113 can acquire operation data input by a user or the like through the input device 200, such as a keyboard, a touch panel, or a touchpad, and can output and display output data to the output device 300, which may include a display device using a liquid crystal or an organic electroluminescence (EL) display. The input device 200 and the output device 300 may be devices built into the information processing device 100, or may be input devices and output devices of other information terminals that can communicate with the information processing device 100 via a network.

[0055] The program memory 111B is a non-transitory tangible storage medium that is a combination of a non-volatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), and a non-volatile memory such as a read only memory (ROM), and can store programs necessary to execute various control processes, etc., according to one embodiment.

[0056] The data memory 112 is a tangible storage medium that is, for example, a combination of the above-mentioned nonvolatile memory and a volatile memory such as RAM (Random Access Memory), and can be used to store various data or information acquired and created during various processes.

[0057] An information processing apparatus 100 according to an embodiment of the present invention can be configured as a data processing apparatus having the units shown in FIG. 1 as software-based processing function units.

[0058] The information storage unit used as a work memory or the like by each unit of the information processing device 100 can be configured by using the data memory 112 shown in Fig. 11. However, these configured storage areas are not essential components within the information processing device 100, and may be areas provided in, for example, an external storage medium such as a USB (Universal Serial Bus) memory, or a storage device such as a database server located in the cloud.

[0059] The processing function units in each of the above units can be realized by reading and executing a program stored in the program memory 111B by the hardware processor 111A. Note that some or all of these processing function units may be realized in various other forms, including integrated circuits such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0060] The methods described in each embodiment can be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, MO, etc.), or semiconductor memory (ROM, RAM, flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only execution programs but also tables and data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-mentioned processing by controlling the operation of this software means. The term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes storage media such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network.

[0061] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0062] DESCRIPTION OF SYMBOLS 100, 100a, 100b, 100c... Information processing device 10... Virtual environment control unit 11... Bottom display control unit 12... Aerial image display control unit 13... Bottom vibration control unit 14... Warmth sensation presentation control unit 20... Touch panel 21... Touch detection unit 22... Bottom display unit 23... Bottom vibration unit 24... Warmth sensation presentation unit 30... Aerial image display unit 31... Light source display 31a... Absorbing polarizing film 32... Beam splitter 32a... Reflecting polarizing film 33... Retroreflective material 33a... 1 / 4 wave plate 34... Mirror

Claims

1. An information processing device comprising: an aerial image display unit that displays an aerial image formed in the air by reflecting light; an image display unit that displays an image of a surface related to the aerial image, which forms the reflective surface of the light reflected by the aerial image display unit; and a control unit that, when an operation is performed to change the content of the image displayed by the image display unit, controls the display content of the aerial image by the aerial image display unit based on the content of the operation.

2. The information processing device of claim 1, wherein when an operation to change the content of the image displayed on the display screen of the image displayed by the image display unit is performed, the control unit controls the vibration of the display screen or controls the temperature of the display screen based on the content of the operation.

3. A method performed by an information processing device, comprising: displaying, by an aerial image display unit of the information processing device, an aerial image formed in the air by reflecting light; displaying, by an image display unit of the information processing device, an image of a surface related to the aerial image, which forms the reflective surface of the light reflected by the aerial image display unit; and, when an operation is performed by a control unit of the information processing device to change the content of the image displayed by the image display unit, controlling the display content of the aerial image by the aerial image display unit based on the content of the operation.

4. An information processing program that causes a processor to function as each part of the information processing device according to claim 1 or 2.

Images