Aerial image display device

Abstract

[Problem] To provide an aerial image display device having same floating sensations regarding sounds as for images.　The aerial image display device forms an image in the air via an optical system so as to realize aerial display. Furthermore, a wavefront synthesis speaker comprising a number of compact speakers is provided to this aerial image display device. A virtual sound source movable in conjunction with the movement of the aerial image is created by the wavefront synthesis speaker at a position that overlaps the aerial display. Consequently, a system is realized that is capable of diverse expressions in both senses of vision and hearing, that accompany floating sensations, with sounds integrated with the movement of images.

Description

[Technical Field]

[0001] The present invention relates to an aerial image display device capable of producing a variety of visual and auditory expressions. [Background technology]

[0002] Interest in aerial video displays has been growing in recent years. One factor behind this is the need for contactless solutions amid the COVID-19 pandemic. Specifically, they are being used in combination with sensors to replace touch panels. For example, at bank ATMs, supermarket self-checkouts, corporate reception desks, and hotel check-ins, an unspecified number of users operate the same touch panel. These touch panels can potentially become a route for infection. To avoid this risk, touch panels need to be disinfected every time a user changes, but such a process is quite time-consuming and difficult to carry out adequately.

[0003] A contactless interface using an aerial image display device can be used with the same ease as a conventional touch panel, without the need for such hassle, and can avoid the risk of infection. For example, in Patent Document 1, since touching a pointing device such as a computer mouse during surgery is a hygiene issue, the surgeon operates a mouse on a contactless remote pointer control device displayed using aerial imaging technology. An example of a special optical element that can be used with such aerial imaging technology is the optical imaging device described in Patent Document 2.

[0004] The way images float in empty space using aerial imaging technology is impactful in itself and helps create a futuristic atmosphere. For this reason, many applications beyond infection prevention measures are conceivable. For example, if applied to digital signage, it can attract more attention. The technology disclosed in Patent Document 3 shows an example of its use in place of barriers at station ticket gates. Furthermore, when displayed as an aerial image, it appears to float above its surroundings, creating a sense of three-dimensionality that is highly entertaining, and there are hopes for its potential application in games and other applications. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2018-147054 [Patent Document 2] International Publication No. 2009 / 131128 [Patent Document 3] Japanese Patent Application Laid-Open No. 2017-142370 Summary of the Invention [Problem to be solved by the invention]

[0006] When displayed as a mid-air image, the displayed object appears to be floating in the air, enabling a highly realistic and immersive experience. On the other hand, in conventional systems, the sound is flat compared to the floating feeling of the image, and the sense of reality and presence cannot be said to be high.

[0007] Therefore, an object of the present invention is to provide an aerial image display device that provides the same floating sensation in terms of sound as in the image. [Means for solving the problem]

[0008] In order to solve the above problems, an aerial image display device according to one aspect of the present invention comprises: In front of the user A floating image display device capable of displaying floating images, which includes a wave field synthesis speaker consisting of a large number of small speakers that configure a virtual sound source to be movable in the air. In front of the user Preparation, The aerial image displays a background and an object as a sound source moving within the background, and the virtual sound source is configured to be superimposed on the aerial image and moves within the background together with the object as the sound source. It is characterized by:

[0009] Also, in one embodiment, Implemented as a video display device for a flight simulator It is characterized by:

[0010] Furthermore, in one embodiment, The piloting simulator is a flight simulator, and the object as the sound source is a flying object flying in the background. It is characterized by: [Effects of the Invention]

[0011] The floating image display device of the present invention enables a variety of visual and auditory expressions, with a floating sensation created by integrating sound with the movement of the image. In particular, when implementing a floating display with a large display area, the virtual sound source can move over a wide range, producing dramatic effects. For example, in a system using a large optical plate exceeding 1 meter in width, larger wave field synthesis speakers can be installed, allowing for a variety of applications that take advantage of the acoustic effects. One such application is the extremely effective use of sound reverberating across a large screen for on-site images such as fireworks. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a perspective view showing a flight simulator 1 using an aerial image display device according to an embodiment of the present invention. [Figure 2] Figure 2 is a diagram explaining the principle by which the display screen of the curved display 20 is imaged as an aerial image G by the optical plate 40 in a flight simulator using an aerial image display device according to an embodiment of the present invention, and shows only the optical plate 40, the display screen of the curved display 20, and the aerial image G of the flight simulator 1 in Figure 1 as viewed from the left side of the flight simulator 1. [Figure 3] Figure 3 is a diagram explaining the principle by which the display screen of the curved display 20 is imaged as an aerial image G by the optical plate 40 in a flight simulator using an aerial image display device according to an embodiment of the present invention, and shows only the optical plate 40, the display screen of the curved display 20, and the aerial image G of the flight simulator 1 in Figure 1 as viewed from directly above the flight simulator 1. [Figure 4] Figure 4 is a diagram explaining the principle by which the display screen of the curved display 20 is imaged as an aerial image G by the optical plate 40 in a flight simulator using an aerial image display device according to an embodiment of the present invention, and shows only the optical plate 40, the display screen of the curved display 20, and the aerial image G of the flight simulator 1 in Figure 1 as viewed from the front of the flight simulator 1. [Figure 5] FIG. 5 is a diagram illustrating the operation of a wave field synthesis speaker in a flight simulator using an aerial image display device according to an embodiment of the present invention. [Figure 6] FIG. 6 is a diagram illustrating a display that replaces the curved display of a flight simulator using an aerial image display device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION [Example]

[0013] Hereinafter, an embodiment of the aerial image display device according to the present invention will be described with reference to the accompanying drawings. In this example, the device is applied to a flight simulator in which a player sits in a cockpit that mimics the real thing, watches images, and controls an aircraft to experience full-scale piloting. In particular, the device is applied to a flight simulator as a competitive game in which players engage in aerial combat with enemy aircraft.

[0014] 1, this flight simulator 1 has a pilot's seat simulating the cockpit of an aircraft, and is made up of a simulator main body 10 and a seat 17 for the pilot who operates this flight simulator 1. The simulator main body 10 is equipped with a control stick 12, control pedals 14, instruments 16, a wave field synthesis speaker 18, etc.

[0015] Mounted inside the simulator main body 10 are a curved display 20, a control device 30, and an optical plate 40. The control device 30 is connected to the control stick 12, control pedals 14, instruments 16, wave field synthesis speaker 18, and curved display 20 via internal wiring (not shown), and exchanges signals with these to simulate flight conditions for a flight experience.

[0016] The incident surface of optical plate 40 faces downward at a certain angle (for example, 45 degrees) toward the display surface of curved display 20. The image on the display surface of curved display 20 is then re-focused as aerial image G at a symmetrical position on the opposite side of optical plate 40, forming the same image as the original. In other words, it can be said that aerial display G is constructed at the imaging position in the air.

[0017] Of course, the images displayed on the curved display 20 are flight images (background, enemy aircraft, etc.) generated by the flight simulator 1. Note that the specific implementation of the flight control simulation performed by the flight simulator 1 here is the same as that of a conventional flight simulator, except for the control of the wave field synthesis speaker 18, which will be described later, and therefore a detailed description thereof will be omitted here.

[0018] The curved display 20 is a curved liquid crystal display device with a convex surface that is placed almost horizontally with the display surface facing upward. Here, the curvature of the convex surface is, for example, 1000R. Instead of a curved liquid crystal display device, a flexible display such as an organic EL display or backlit electronic paper may be curved to the desired curvature. In either case, it is important that the display surface is convex upward.

[0019] Here, the position of curved display 20 is fixed, but the support structure may be designed so that its position can be adjusted in the vertical direction. In this case, the focusing position of aerial image G can be adjusted to a position that is easy for the pilot to see.

[0020] The control device 30 is essentially a small computer and is composed of a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), a storage device for storing various programs and data, an input / output interface, etc. Examples of the input / output interface include a USB port and a wireless LAN such as Wi-Fi. Various data related to the flight simulation and program updates can be performed via this input / output interface. The control device 30 outputs a video signal to the curved display 20 to display the source of the aerial image, and also outputs a drive signal to the wave field synthesis speaker 18 to reproduce a sound field synchronized with the image on the curved display 20.

[0021] The optical plate 40 may be, for example, an optical imaging element (two-sided orthogonal reflector) as described in Japanese Patent Laid-Open Publication No. 2011-175297. This optical imaging element is realized by arranging a large number of mutually orthogonal planar light reflecting portions at a fixed pitch. Alternatively, a two-sided corner reflector, in which reflective surfaces are formed on the side surfaces of a square hole as described in Japanese Patent No. 4900618, may be used.

[0022] Figure 2 is a diagram explaining the principle by which the display screen of the curved display 20 is imaged as an aerial image G by the optical plate 40. For simplicity of explanation, only the optical plate 40 of the flight simulator 1 in Figure 1, the display screen of the curved display 20, and the aerial image G are shown as seen from the left side of the flight simulator 1.

[0023] Figure 3 also explains the principle by which the display screen of the curved display 20 is imaged as an aerial image G by the optical plate 40, and for simplicity of explanation, only the optical plate 40 of the flight simulator 1 in Figure 1, the display screen of the curved display 20, and the aerial image G are shown as a view from directly above the flight simulator 1.

[0024] Furthermore, Figure 4 is also a diagram explaining the principle by which the display screen of the curved display 20 is imaged as an aerial image G by the optical plate 40, and for the sake of simplicity, only the optical plate 40 of the flight simulator 1 in Figure 1, the display screen of the curved display 20, and the aerial image G are shown as seen from the front of the flight simulator 1 (behind the seat 17 in Figure 1).

[0025] Under certain conditions of incident light, optical plate 40 with a dual reflection structure such as a two-sided orthogonal reflector or a two-sided corner reflector retroreflects incident light in the direction of the panel plane without changing the component of the incident light orthogonal to the panel plane. As a result, the display screen of curved display 20 and aerial image G are plane-symmetric with respect to optical plate 40.

[0026] That is, position L1 (the most convex position) in the horizontal center of the curved display 20 is closest to the optical plate 40, and light emitted from position L1 is focused at position M1 closest to the optical plate 40. In other words, light emitted from position L1 is reflected at arbitrary positions R1, R1 on the optical plate 40, and focused at position M1 on the opposite side of the optical plate 40, a distance equal to the distance between position L1 and the optical plate 40.

[0027] Similarly, position L2 on the lateral outer side of the curved display 20 is distant from the optical plate 40, and light emitted from position L2 is focused at position M2 distant from the optical plate 40. In other words, light emitted from position L2 is reflected at any position R2, R2 on the optical plate 40, and focused at position M2 on the opposite side of the optical plate 40, a distance equal to the distance between position L2 and the optical plate 40.

[0028] As a result, from the perspective of the pilot sitting in seat 17, the aerial image G displayed in front of him / her appears to be curved concavely, and the concave curved aerial display is implemented as if it were floating in the air.

[0029] In flight simulators using conventional flat displays, the left and right edges of the display are far from the eyes, making them difficult to see, and the image is distorted at the edges, resulting in a significant discrepancy from the actual field of view.In contrast, this concave curved aerial display can seamlessly reproduce the field of view seen by the pilot during an actual flight.

[0030] Furthermore, since there is no physical display in the air, the air display creates a three-dimensional, more realistic piloting experience. Furthermore, with physical displays, the reflection of external light on the display surface can inevitably obstruct the pilot's view, but with the air display, such reflection of external light is not possible, allowing for a deeply immersive simulated piloting experience without being disturbed by external light.

[0031] Next, we will explain the wave field synthesis speaker 18, a component characteristic of the present invention. The wave field synthesis speaker 18 is composed of multiple small speakers 18-1, 18-2, ... 18-32 arranged in a line, and each small speaker is controlled separately by the control device 30. The more small speakers there are, the more accurate the sound field reproduction will be, but in this example, a line speaker with 32 small speakers arranged horizontally is used. In general, the desired number of small speakers as a component of the wave field synthesis speaker 18 is at least 8, and preferably 16 or more.

[0032] The control device 30 controls the small speakers 18-1, 18-2, ..., 18-32 using wave field synthesis (WFS) technology, and a virtual sound source is realized at a predetermined position. Here, realizing a virtual sound source means, based on Huygens' principle, creating a sound field in an assumed listening area (here, near the pilot's head) so that an actual sound source exists at the position of the virtual sound source.

[0033] Therefore, the program routine for controlling the speakers of the control device 30 includes not only audio data such as engine sounds, but also position data for specifying the position at which the audio data is to be reproduced.

[0034] Figure 5 is a diagram explaining the virtual sound source generated by the wave field synthesis speaker 18 of the present invention. For simplicity of explanation, only the wave field synthesis speaker 18 and the aerial display (aerial image) G of the flight simulator 1 in Figure 1 are shown as a view from directly above the flight simulator 1.

[0035] As shown in Figure 5, by controlling the individual small speakers 18-1, 18-2, .... 18-32 using wave field synthesis technology, the individual sound waves are superimposed to synthesize the wave front W of the virtual sound source V. In reality, it is sufficient that the virtual sound source V is recognized by the pilot P, so it is sufficient that the wave front W of the virtual sound source V is reproduced only in front of the aerial display G.

[0036] In this application to a competitive game, the virtual sound source V is a sound source that reproduces the engine sounds of the enemy aircraft. The virtual sound source V is set to the position of the enemy aircraft displayed on the aerial display G, and moves in accordance with the movement of the enemy aircraft on the screen, as shown by the arrow in Figure 5. The wavefront W is faithfully synthesized, so the Doppler effect is also reproduced.

[0037] In FIG. 5, the virtual sound source V that reproduces the engine sound of the enemy aircraft moves on the aerial display G (light-collecting surface), but this is not limiting. The virtual sound source V may be controlled to jump out in front of or behind the aerial display G depending on the movement of the enemy aircraft. For example, in a scene where the enemy aircraft is approaching and passing by the pilot's aircraft, the virtual sound source V may be moved behind the pilot P. In such a case, in order to further emphasize the Doppler effect, the frequency of the engine sound of the approaching enemy aircraft may be made slightly higher and the frequency of the engine sound of the receding enemy aircraft may be made slightly lower, thereby making the sense of realism more effective.

[0038] Furthermore, wave field synthesis technology allows the synthesis of multiple virtual sound sources, making it possible to reproduce the engine sounds of multiple enemy aircraft separately. In other words, a virtual sound source is reproduced in one-to-one correspondence with each enemy aircraft. This allows the pilot to sense the presence of multiple enemy aircraft not only visually but also aurally. Furthermore, by placing a virtual sound source V' behind the pilot that reproduces the engine sound of the pilot's aircraft, the sense of realism is further enhanced.

[0039] Generally, when synthesizing a virtual sound source using a wave field synthesis speaker 18, if there is an object such as a display near the position of the virtual sound source, the sound waves will interfere with the object, and synthesis will not work well. However, in this invention, since the aerial display has no physical substance, it is possible to overlay the virtual sound source on the display or move the virtual sound source in front of or behind the display, crossing it. This enables a variety of expressions in both visual and auditory senses.

[0040] Although the flight simulator 1 of the above embodiment can seamlessly display a concave curved aerial image, it requires a dedicated curved display, which can be costly. Instead, using a combination of multiple flat displays can further reduce manufacturing costs.

[0041] 6, a central flat display 64-2 may be placed almost horizontally with its display surface facing upward, and flat displays 64-1 and 64-3 may be placed on either side of it at a fixed angle (for example, 20 degrees) in close contact with each other. In this case, by combining an inexpensive general-purpose liquid crystal display, a display that is concave overall, although not a smoothly curved surface as in the above embodiment, can be realized at low manufacturing cost. [Industrial Applicability]

[0042] The floating image display device according to the present invention makes it possible to create a variety of visual and auditory expressions, with a floating sensation achieved by integrating sound with the movement of the image.

[0043] The above describes the aerial image display device according to the present invention based on an embodiment, but the present invention is not limited to this, and modifications may be made within the scope of the spirit of the present invention, and if possible, the techniques described in each embodiment may be combined, or publicly known techniques may be combined, etc.

[0044] For example, in the above embodiment, the aerial image display device is applied to a piloting simulator that simulates a cockpit, but the application of the present invention is not limited to this. It can also be applied to various competitive simulators, such as war simulation games set on land or in space, and racing simulators. Furthermore, if it is applied to a web conferencing system in which multiple participants are displayed on the screen, and a virtual sound source is assigned to each participant and audio is output from the displayed position of that participant, it can create a more realistic feeling as if the participant were participating in a real conference.

[0045] In the above embodiment, a row of line speakers is provided as a wave field synthesis speaker on the upper side of the optical plate 40, but the present invention is not limited to this. For example, multiple line speakers may be provided on the upper and lower sides of the optical plate 40, or on the left and right sides.

[0046] Furthermore, typical aerial image display devices often incorporate a contactless interface that allows operation by touching the aerial image. Since the above embodiment simulates a cockpit, such a contactless interface is omitted. However, depending on the application, a contactless interface that allows operation by touching the aerial image can function very effectively. In such cases, as with conventional aerial image display devices, a contactless interface can be implemented by providing an operation detection unit consisting of an infrared LED and an infrared camera near the aerial image (for example, on the front side of optical plate 40).

[0047] Furthermore, in the above-described embodiment, a curved aerial image is realized using a curved display, but this is costly. Alternatively, a single, general-purpose flat LCD display may be used. In this case, the aerial image will also be flat, but this does not pose any problems for implementing the present invention.

[0048] Furthermore, in the above embodiment, an optical system using an optical plate with a double reflection structure, such as a two-sided orthogonal reflector or a two-sided corner reflector, is used as the optical system, but the present invention is not limited to this and can be applied to other types of aerial image display devices. For example, a retroreflection system may be adopted in which a beam splitter is used as the optical plate 40 and a retroreflector plate is provided on the back side. [Explanation of symbols]

[0049] 1. Flight Simulator 10 Simulator body 12 control sticks 14 Control pedals 16 Instruments 17 seats 18 Wave Field Synthesis Speaker 20 Curved Display 30 Control device 40 Optical Plate 64-1, 64-2, 64-3 Flat Panel Displays

Claims

1. An aerial image display device capable of displaying an aerial image in a direction directly in front of a user, A wave field synthesis speaker consisting of many small speakers that create a virtual sound source that can be moved in the air is provided in front of the user. The aerial image displays a background and an object as a sound source moving within the background, The aerial image display device is characterized in that the virtual sound source is configured to be superimposed on the aerial image and moves within the background together with an object that is the source of the sound.

2. An aerial video display device as described in claim 1, characterized in that it is implemented as a video display device for a piloting simulator.

3. The aerial image display device described in Claim 2, characterized in that the piloting simulator is a flight simulator and the object that is the source of the sound is a flying object flying within the background.