Aerial image display device

Abstract

To provide an aerial image display device for easily detecting the position of a user's finger with respect to an aerial image.SOLUTION: An aerial image display device is provided with a casing 2, an image display unit 3 disposed inside the casing, a beam splitter 4 installed in an opening of the casing for reflecting an image displayed on the image display unit, a retroreflective sheet 5 installed inside the casing for retroreflecting the image reflected by the beam splitter, a sensor 7 for detecting the position of a finger 10a of a user 10 within an aerial region containing an aerial image 6 obtained by image-forming an image retroreflected by the retroreflective sheet outside the casing, and a stereo speaker 9 for outputting sound whose localization changes in accordance with the position of the finger detected by the sensor with respect to the aerial image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This application relates to an aerial video display device.

Background Art

[0002] There is known an aerial video display device that does not require special glasses and can display a video in the air without using a screen such as fog. For example, there is disclosed an aerial video display device that uses a beam splitter and a retroreflective sheet to form an image of the video displayed on the video display unit in a space at a position symmetric with respect to the beam splitter (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In an aerial video display device, there is a method of using it as an aerial touch panel in which a touch panel is displayed as a video displayed in the air (hereinafter referred to as an aerial video), and a user operates the touch panel with a finger. Since the aerial video floats in the air, there is no object around it that serves as an indicator of the position of the user's finger. Therefore, there is a problem that it is difficult for the user to grasp the position of their finger with respect to the aerial video.

[0005] This application has been made to solve the above problems, and an object thereof is to provide an aerial video display device that makes it easy for a user to grasp the position of their finger with respect to an aerial video.

Means for Solving the Problems

[0006] The aerial video display device of this application is Display an image a video display unit and , image the video displayed on the image display unitAn aerial imaging optical system that forms an image in space to display an aerial image, A sensor that detects the position of a user's finger within a spatial region containing an aerial image, and a stereo speaker that outputs a sound whose localization changes according to the position of the detected finger position with respect to the aerial image.

Advantages of the Invention

[0007] Since the aerial image display device of the present application is provided with a stereo speaker that outputs a sound whose localization changes according to the position of the finger position detected by the sensor with respect to the aerial image, it becomes easy to grasp the position of the user's finger with respect to the aerial image.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Modes for Carrying Out the Invention

[0009] Hereinafter, an aerial image display device according to an embodiment for carrying out the present application will be described in detail with reference to the drawings.

[0010] Embodiment 1. FIG. 1 is a cross-sectional view of an aerial video display device according to Embodiment 1. FIG. 2 is a schematic diagram of the aerial video display device according to the present embodiment. The aerial video display device 1 of the present embodiment includes a housing 2, a video display unit 3 disposed inside the housing 2, a beam splitter 4 provided at an opening of the housing 2, and a retroreflective sheet 5 disposed inside the housing 2. The light emitted from the video display unit 3 enters the beam splitter 4, and a part of it is reflected and enters the retroreflective sheet 5. The retroreflective sheet 5 has the property of reflecting the incident light in the incident direction. The retroreflective sheet 5 reflects the light incident from the beam splitter 4 toward the beam splitter 4. A part of the light reflected from the retroreflective sheet 5 toward the beam splitter 4 passes through the beam splitter 4. In this way, the video displayed on the video display unit 3 passes through the beam splitter 4 and forms an image outside the housing 2 to become an aerial video 6. The user 10 can observe this aerial video 6. For example, a touch panel for operating various devices is displayed as the aerial video 6. The user 10 can perform a touch operation on the touch panel by bringing the finger 10a close to the touch panel displayed as the aerial video 6. The position of the finger 10a can be measured by a three-dimensional distance sensor 7 provided on the housing 2 and detecting the position of an object within the spatial region including the aerial video 6. As the video display unit 3, a liquid crystal display, an organic EL display, a display in which LED elements are arranged, etc. can be used. As the beam splitter 4, a half mirror, a reflective polarizing plate, etc. can be used. As the retroreflective sheet 5, a glass bead type sheet in which spherical transparent glass beads are arranged, a micro prism type sheet in which transparent triangular prisms are arranged, etc. can be used. As the three-dimensional distance sensor 7, for example, a ToF sensor (Time of Flight Sensor), a stereo camera, etc. can be used. As shown in FIG. 1 here, a direction perpendicular to the aerial video 6 is defined as the z-axis direction, the horizontal direction of the aerial video 6 is defined as the x-axis direction, and a direction orthogonal to the z-axis direction and the x-axis direction is defined as the y-axis direction. Hereinafter, the x-axis direction is also referred to as the left-right direction, the y-axis direction is also referred to as the up-down direction, and the z-axis direction is also referred to as the perpendicular direction.

[0011] The user 10 can perform a touch operation by bringing the finger 10a close to the touch panel displayed as the aerial image 6. The positional relationship between the position of the finger 10a and the position of the aerial image 6 can be measured by the three-dimensional distance sensor 7 provided in the housing 2. Since this touch operation is non-contact, it is possible to prevent contamination of the fingertips due to contact, diffusion of harmful substances through the touch panel, and the like.

[0012] However, since the aerial image floats in the air, there is no object around it that serves as an indicator of the position of one's own finger. Therefore, there is a problem that it becomes difficult for the user to grasp the position of their own finger with respect to the aerial image.

[0013] The aerial image display device 1 of the present embodiment is such that it is easy for the user to grasp the position of their own finger with respect to the aerial image. Therefore, the aerial image display device 1 of the present embodiment includes a control unit 8 that calculates the position of the finger 10a of the user 10 detected by the three-dimensional distance sensor 7 with respect to the aerial image 6, and a stereo speaker 9 connected to the control unit 8. The stereo speaker 9 is composed of two speakers arranged along the x-axis direction as shown in FIG. 2. The control unit 8 calculates the position of the finger 10a of the user 10 with respect to the aerial image 6, and causes the stereo speaker 9 to output a sound whose localization changes according to the position. Here, localization refers to the direction of the sound source, that is, the direction from which the sound is heard, in the sound field of the stereo reproduced sound.

[0014] When the distance between the position of the finger 10a of the user 10 in the z-axis direction and the aerial image 6 becomes smaller than a predetermined threshold value with respect to the aerial image 6, the control unit 8 starts outputting a low-frequency bass sound from the stereo speaker 9. At this time, the control unit 8 determines the sound localization of the sound output from the stereo speaker 9 according to the position of the finger 10a of the user 10 with respect to the aerial image 6 in the x-axis direction. For example, when the position of the finger 10a of the user 10 is close to the central part of the aerial image 6, the control unit 8 sets the sound localization to the center, and when the position of the finger 10a of the user 10 moves to the right side of the aerial image 6, the control unit 8 moves the sound localization to the right side. Further, the control unit 8 determines the volume of the sound output from the stereo speaker 9 according to the position of the finger 10a of the user 10 with respect to the aerial image 6 in the y-axis direction. For example, when the position of the finger 10a of the user 10 is close to the central part of the aerial image 6, the control unit 8 sets the volume to medium volume, and when the position of the finger 10a of the user 10 moves to the lower side of the aerial image 6, the control unit 8 increases the volume. Also, as the distance between the position of the finger 10a of the user 10 in the z-axis direction and the aerial image 6 becomes smaller, the control unit 8 gradually increases the frequency and changes the sound output from the stereo speaker 9 to a higher pitch.

[0015] In the aerial image display device configured as described above, the user can recognize the distance in the direction perpendicular to the position of their finger and the aerial image by the change in the frequency of the sound output from the stereo speaker. Also, the user can recognize the left-right position of their finger with respect to the aerial image by the change in the sound localization of the sound output from the stereo speaker. Further, the user can recognize the up-down position of their finger with respect to the aerial image by the change in the volume of the sound output from the stereo speaker. Therefore, in the aerial image display device of the present embodiment, it becomes easy for the user to grasp the position of their finger with respect to the aerial image. Also, since the sound changes diversely with the movement of the finger, the user can obtain a feeling of operating the aerial image display device, thus improving the operating feeling.

[0016] The control unit 8 may discontinuously change the frequency of the sound output from the stereo speaker 9 when the position of the finger and the position of the aerial image coincide. As another method, the control unit 8 may stop the sound output from the stereo speaker 9 when the position of the finger and the position of the aerial image coincide. By controlling in this way, the user can recognize that their finger has reached the position of the aerial image. Note that the case where the position of the finger and the position of the aerial image coincide is not limited to the case where the separation distance between the position of the finger and the position of the aerial image becomes exactly zero, and also includes the case where it is smaller than a predetermined separation distance. Further, the control unit 8 may emit a special sound as a touch sound when the user's finger touches the button of the touch panel displayed in the aerial image. Note that when the user's finger passes through the aerial image, the frequency of the sound output from the stereo speaker 9 decreases, so the user can also recognize that their finger has passed beyond the position of the aerial image. Further, when the user's finger passes through the aerial image, the sound output from the stereo speaker 9 may be a dissonant sound.

[0017] Note that in the present embodiment, the control unit 8 gradually increases the frequency as the separation distance in the right-angled direction decreases, and changes the sound output from the stereo speaker to a higher pitch. However, the frequency may be gradually decreased as the separation distance decreases, and the sound output from the stereo speaker may be changed to a lower pitch. Further, the control unit 8 increases the volume as the position of the finger 10a of the user 10 moves downward with respect to the aerial image 6. However, the volume may be decreased as the position of the finger 10a of the user 10 moves downward with respect to the aerial image 6.

[0018] Embodiment 2. In the aerial video display device according to Embodiment 1, the control unit causes the stereo speaker to output a sound whose frequency continuously changes according to the separation distance in the direction perpendicular to the position of the user's finger and the aerial video. In the aerial video display device according to Embodiment 2, the control unit causes the stereo speaker to output a sound in which intermittent sounds continuously change according to the separation distance in the direction perpendicular to the position of the user's finger and the aerial video. Note that the configuration of the aerial video display device of the present embodiment is the same as the configuration of the aerial video display device of Embodiment 1 shown in FIGS. 1 and 2. Further, the control unit, as in Embodiment 1, changes the sound localization output from the stereo speaker according to the position of the user's finger in the horizontal direction with respect to the aerial video, and changes the volume of the sound output from the stereo speaker according to the position of the user's finger in the vertical direction with respect to the aerial video.

[0019] For example, when the finger 10a of the user 10 approaches the aerial video 6 and the separation distance in the z-axis direction becomes smaller than a predetermined threshold value, the control unit 8 starts outputting intermittent sounds from the stereo speaker 9. At this time, the sound output from the stereo speaker 9 is a sound in which a sound of a certain length is generated with a stop period in between, and the control unit 8 gradually shortens the stop period as the separation distance becomes smaller. Further, the control unit 8, as in Embodiment 1, changes the sound localization output from the stereo speaker 9 according to the position of the finger 10a of the user 10 in the x-axis direction with respect to the aerial video 6 and the position of the aerial video 6. Furthermore, the control unit 8 changes the volume of the sound output from the stereo speaker 9 according to the position of the finger 10a of the user 10 in the y-axis direction with respect to the aerial video 6 and the position of the aerial video 6.

[0020] In the aerial video display device configured as described above, the user can recognize the separation distance in the direction perpendicular to the position of their finger and the aerial video by the change in the silence period of the sound output from the speaker. Also, the user can recognize the position of their finger in the left-right direction with respect to the aerial video by the change in the sound localization of the sound output from the stereo speakers. Further, the user can recognize the position of their finger in the up-down direction with respect to the aerial video by the change in the volume of the sound output from the stereo speakers. Therefore, in the aerial video display device of the present embodiment, it becomes easier for the user to grasp the position of their finger with respect to the aerial video. Also, since the sound changes diversely with the movement of the user's finger, the user can obtain a feeling of operating the aerial video display device, thus improving the operating feeling.

[0021] Note that in the present embodiment, as the sound output from the stereo speaker 9, a sound having a certain length is generated with a silence period in between, and the silence period gradually becomes shorter as the separation distance in the perpendicular direction becomes smaller. As another method, the silence period may be fixed, and the length of the generation period during which the sound is generated may gradually become shorter as the separation distance becomes smaller. As yet another method, the period for starting the generation of intermittent sound may be fixed, and the period during which the sound is generated within that period may be gradually changed as the separation distance becomes smaller. That is, in the aerial video display device of the present embodiment, at least one of the silence period and the generation period of the intermittent sound continuously changes according to the separation distance in the perpendicular direction between the position of the user's finger and the position of the aerial video.

[0022] The control unit 8 may discontinuously change at least one of the stop period and the generation period of the sound output from the stereo speaker 9 when the position of the finger and the position of the aerial image coincide. As another method, the control unit 8 may change the frequency of the sound output from the stereo speaker 9 when the position of the finger and the position of the aerial image coincide. As yet another method, the control unit 8 may stop the sound output from the stereo speaker 9 when the position of the finger and the position of the aerial image coincide. By controlling in this way, the user can recognize that their finger has reached the position of the aerial image. Note that the time when the position of the finger and the position of the aerial image coincide is not limited to when the separation distance between the position of the finger and the position of the aerial image becomes exactly zero, and also includes cases where it is smaller than a predetermined separation distance. Further, the control unit 8 may emit a special sound as a touch sound when the user's finger touches a button on the touch panel displayed in the aerial image. Note that when the user's finger passes through the aerial image, the stop period or the generation period of the sound output from the stereo speaker 9 becomes longer, so the user can also recognize that their finger has passed beyond the position of the aerial image. Also, when the user's finger passes through the aerial image, the sound output from the stereo speaker 9 may be a dissonant sound.

[0023] Embodiment 3. In the aerial image display devices of Embodiments 1 and 2, the control unit changes the sound localization of the sound output from the stereo speaker according to the position of the user's finger and the aerial image in the x-axis direction, and changes the volume of the sound output from the stereo speaker according to the position of the user's finger and the aerial image in the y-axis direction. In the aerial image display device of Embodiment 3, a stereo speaker array is used to change the sound localization in the xy plane output from the stereo speaker array according to the position of the user's finger in the xy plane.

[0024] FIG. 3 is a schematic diagram of the aerial video display device according to the present embodiment. The aerial video display device 1 of the present embodiment is the aerial video display device of Embodiment 1, and includes a pair of stereo speaker arrays 11 as stereo speakers. As shown in FIG. 3, the stereo speaker array 11 is composed of two speaker arrays arranged along the x-axis direction. The plurality of speakers constituting each speaker array are arranged along a direction parallel to the y-axis direction. Therefore, the stereo speaker array 11 can change the sound localization in the xy plane.

[0025] When the distance between the position of the finger 10a of the user 10 in the z-axis direction with respect to the aerial video 6 and the aerial video 6 becomes smaller than a predetermined threshold value, the control unit 8 starts outputting a low-frequency bass sound from the stereo speaker array 11. At this time, the control unit 8 changes the sound localization in the xy plane of the sound output from the stereo speaker array 11 according to the position of the finger 10a of the user 10 in the xy plane with respect to the aerial video 6. When the position of the finger 10a of the user 10 is close to the central portion of the aerial video 6, the control unit 8 sets the sound localization to the center, and when the position of the finger 10a of the user 10 moves in the xy plane of the aerial video 6, the control unit 8 moves the sound localization according to that position. In addition, as the distance between the position of the finger 10a of the user 10 in the z-axis direction with respect to the aerial video 6 and the aerial video 6 becomes smaller, the control unit 8 gradually increases the frequency and changes the sound output from the stereo speaker 9 to a high pitch.

[0026] Note that the control unit 8 changes the frequency according to the distance between the position of the finger 10a of the user 10 in the z-axis direction with respect to the aerial video 6 and the aerial video 6, but instead of changing the frequency, the intermittent sound period may be changed. The control unit 8 of the aerial video display device of the present embodiment changes the sound quality of the sound output from the stereo speaker array 11 according to the distance between the position of the finger 10a of the user 10 in the z-axis direction with respect to the aerial video 6 and the aerial video 6.

[0027] In the airborne video display device configured as described above, the user can recognize the separation distance in the z-axis direction between the position of the airborne video and the position of their finger by the change in the sound quality of the sound output from the speaker. Also, the user can recognize the position of their finger in the xy plane with respect to the airborne video by the change in the sound localization in the xy plane of the sound output from the stereo speaker array. Therefore, in the airborne video display device of the present embodiment, it becomes easier for the user to grasp the position of their finger with respect to the airborne video. Also, since the sound changes diversely with the movement of the finger, the user can obtain a feeling of operating the airborne video display device, improving the operability.

[0028] In the present embodiment, the sound localization in the xy plane is changed using a stereo speaker array, but three or more speakers may be used instead of the stereo speakers to change the sound localization in the xy plane.

[0029] In Embodiments 1 to 3, the position of the finger 10a is measured by the three-dimensional distance sensor 7. The airborne video display devices of Embodiments 1 to 3 may include a recognition unit for calculating the position of the fingertip that is the position to touch the touch panel from the position of the finger 10a measured by the three-dimensional distance sensor 7. Also, in Embodiments 1 to 3, it is also assumed that the display image is switched according to the touch operation when using the airborne video as a touch panel. The airborne video display devices of Embodiments 1 to 3 may include a display processing unit for switching the display image of the video display unit based on the position of the fingertip calculated by the recognition unit.

[0030] In addition, in Embodiments 1 to 3, an airborne video display device using a beam splitter and a retroreflective sheet was described. As other types of airborne video display devices, there are those using an optical plate called a two-sided corner reflector array and those using an optical plate called an orthogonal mirror array. In these types of airborne video display devices, in the airborne video display device shown in FIG. 1, the retroreflective sheet is removed and an optical plate is arranged at the position of the beam splitter. Such an airborne video display device can display an airborne video by forming an image of the video displayed on the video display unit in a space at a position that is axially symmetric with respect to the optical plate. Even in such an airborne video display device, there is a problem that it is difficult to grasp the position of the user's finger with respect to the airborne video. Even in such an airborne video display device, by providing a stereo speaker that outputs a sound whose localization changes according to the position of the user's finger with respect to the airborne video, it becomes easier to grasp the position of the user's finger with respect to the airborne video.

[0031] Although various exemplary embodiments and examples are described in this application, the various features, aspects, and functions described in one or more of the embodiments are not limited to the application of a specific embodiment, but are applicable to the embodiments alone or in various combinations. Therefore, countless variations not illustrated are envisioned within the scope of the technology disclosed in this application specification. For example, it includes cases where at least one component is modified, added, or omitted, and further cases where at least one component is extracted and combined with components of other embodiments.

Description of Reference Numerals

[0032] 1 Airborne video display device, 2 Housing, 3 Video display unit, 4 Beam splitter, 5 Retroreflective sheet, 6 Airborne video, 7 Three-dimensional distance sensor, 8 Control unit, 9 Stereo speaker, 10 User, 10a Finger, 11 Stereo speaker array.

Claims

1. An aerial video display device comprising: a video display unit for displaying a video; an aerial imaging optical system for imaging the video displayed on the video display unit in space to display an aerial video; a sensor for detecting the position of a user's finger within a spatial region containing the aerial video; and a stereo speaker for outputting a sound whose localization changes according to the position of the finger detected by the sensor with respect to the aerial video.

2. The aerial video display device according to claim 1, wherein the aerial imaging optical system comprises a beam splitter for reflecting the video displayed on the video display unit, and a retroreflective sheet for retroreflecting the video reflected by the beam splitter.

3. The aerial video display device according to claim 1, wherein the aerial imaging optical system is composed of an optical plate for imaging the video displayed on the video display unit in space to display the aerial video, and the optical plate images the video displayed on the video display unit in a space at a position that is plane-symmetric with respect to the optical plate to display the aerial video.

4. The aerial video display device according to any one of claims 1 to 3, further comprising a control unit for controlling the sound output from the stereo speaker according to the position of the finger detected by the sensor with respect to the aerial video, wherein the control unit changes the sound quality of the sound output from the stereo speaker according to the separation distance between the position of the finger in a direction perpendicular to the aerial video and the aerial video, and changes the localization of the sound output from the stereo speaker according to the position of the finger in a direction parallel to the aerial video with respect to the aerial video.

5. The aerial video display device according to claim 4, wherein the frequency of the sound output from the stereo speaker continuously changes according to the separation distance between the position of the finger in a direction perpendicular to the aerial video and the aerial video.

6. The sound output from the stereo speaker is characterized in that the frequency changes discontinuously when the position of the finger coincides with the position of the aerial image, for the aerial image display device according to claim 5.

7. The sound output from the stereo speaker is an intermittent sound, and at least one of the stop period and the generation period of the intermittent sound changes continuously according to the separation distance between the position of the finger and the aerial image in a direction perpendicular to the aerial image, for the aerial image display device according to claim 4.

8. The sound output from the stereo speaker is characterized in that at least one of the stop period and the generation period changes discontinuously when the position of the finger coincides with the position of the aerial image, for the aerial image display device according to claim 7.

9. The sound output from the stereo speaker is characterized in that it stops when the position of the finger coincides with the position of the aerial image, for the aerial image display device according to claim 5 or 7.

10. The sound output from the stereo speaker is characterized in that it becomes a dissonant sound when the position of the finger passes through the aerial image, for the aerial image display device according to any one of claims 4 to 9.

11. The control unit changes the sound localization of the sound output from the stereo speaker according to the position of the finger with respect to the aerial image in one direction parallel to the aerial image, and changes the volume of the sound output from the stereo speaker according to the position of the finger with respect to the aerial image in a direction perpendicular to the one direction, for the aerial image display device according to any one of claims 4 to 10.

12. The stereo speaker is composed of a pair of speaker arrays, for the aerial image display device according to any one of claims 1 to 11.

Images