Aerial image display device

Abstract

An aerial image display device (1) according to the present disclosure comprises a light source (24), an optical system (30) that forms an aerial image of light emitted by the light source (24), and a rotating mirror (40) positioned between the light source (24) and the optical system (30).

Description

Aerial image display device

[0001] This application claims priority to U.S. patent application Ser. No. 63 / 692,183, filed Sep. 9, 2024, the contents of which are incorporated herein by reference.

[0002] Conventionally, aerial images have been displayed using retrotransmissive optical elements (see, for example, Patent Document 1).

[0003] Japanese Patent Application Publication No. 2021-139966

[0004] However, the aerial image display device described in Patent Document 1 is unable to display an aerial image in accordance with the movement of the user's head.

[0005] The present disclosure has been made in consideration of the above-mentioned circumstances, and aims to provide an aerial image display device that can form an aerial image according to the orientation of the user's head.

[0006] In order to solve the above problems, the present disclosure proposes the following means: An aerial image display device according to one aspect of the present disclosure includes a light source, an optical system that forms an aerial image of light emitted by the light source, and a rotating mirror positioned between the light source and the optical system.

[0007] According to the present disclosure, it is possible to provide an aerial image display device that can form an aerial image according to the orientation of the user's head.

[0008] FIG. 1 is a side view of an aerial image display device according to an embodiment of the present invention; FIG. 2 is a front view of an aerial image display device according to an embodiment of the present invention; FIG. 3 is a schematic plan view of an aerial image display device according to an embodiment of the present invention, showing light emitted from a light source; FIG. 4 is a schematic side view of an aerial image display device according to an embodiment of the present invention, showing an optical path connecting an aerial image; FIG. 5 is a schematic block diagram for explaining a control device of an aerial image display device according to an embodiment of the present invention; FIG. 6 is a schematic block diagram showing a specific example of a control device of an aerial image display device according to an embodiment of the present invention; FIG. 7 is a schematic plan view of an aerial image display device according to an embodiment of the present invention, showing light emitted from a light source; FIG. 8 is a schematic front view of an aerial image display device according to an embodiment of the present invention, showing movement of an aerial image.

[0009] An aerial image display device according to one embodiment of the present disclosure will be described below with reference to the drawings. In recent years, with the widespread use of HMDs (head-mounted displays), users have been exploring various ways of using VR spaces. Users can view images (video) by wearing an HMD on their head. However, when a user views a video in a supine position, such as when experiencing content such as "VR sleep," wearing an HMD can cause issues such as a feeling of pressure on the face and difficulty in turning over. Therefore, there has been a need to reduce the burden on users when viewing HMD images in a supine position.

[0010] (Embodiment) The following will first describe the configuration of an aerial image display device 1. As shown in Figures 1 and 2, the aerial image display device 1 includes a housing 20, a light source 24, an optical system 30, and a rotating mirror 40. The housing 20 is a member that mainly forms the base of the aerial image display device 1. The light source 24, the optical system 30, and the rotating mirror 40 are arranged on top of the housing 20.

[0011] As shown in FIG. 1 , the housing 20 includes legs 21, a frame body 22, and a support 23. The legs 21 are members extending in the vertical direction. The number of legs 21 is, for example, four, but may be five or more. The frame body 22 is, for example, a rectangular member. The frame body 22 is supported by the legs 21. The user 80 can enter below the frame body 22 while lying supine. The support 23 is a member for supporting each component of the aerial image display device 1 arranged on the housing 20. A plurality of support parts 23 are provided. One end of the support part 23 is fixed to the frame body 22.

[0012] As shown in Fig. 1, the light source 24 is, for example, an LED, but is not particularly limited to this. Any known light source can be suitably used as the light source 24. In the example shown, the light source 24 is provided in a known HMD 25. The HMD 25 includes an eyepiece 26. The light source 24 emits light from, for example, the eyepiece 26. The eyepiece 26 is arranged to face the direction of the rotating mirror 40. The light from the light source 24 emitted from the eyepiece 26 is directed toward the rotating mirror 40.

[0013] As shown in FIG. 1 , the rotating mirror 40 emits light incident from the light source 24 to the optical system 30. The rotating mirror 40 has a rectangular shape that is long in the vertical direction. As shown in FIG. 2 , the rotating mirror 40 has a rotation axis O1 and a motor (not shown). The rotation axis O1 is an axis along the longitudinal direction of the rotating mirror 40. The rotation axis O1 is along the front-to-back direction of the user 80 in a supine position. In the example shown, the rotation axis O1 is provided at the center of the rotation mirror 40 in the short direction. The rotating mirror 40 can be rotated around the rotation axis O1 by, for example, a motor.

[0014] 1 , the optical system 30 includes, for example, a retrotransmissive optical element 31 and two mirrors 32. Note that hereinafter, the optical system 30 including the retrotransmissive optical element 31 and two mirrors 32 may be referred to as a TRISFER. The optical system 30 according to this embodiment is characterized in that depth inversion does not occur throughout the entire optical system 30.

[0015] The retrotransmitting optical element 31 is an optical element that can form an aerial image I3. Known retrotransmitting optical elements can be suitably used for the retrotransmitting optical element 31. The retrotransmitting optical element 31 is, for example, a dual SMAs. For example, the retrotransmitting optical element 31 is a plate in which a plurality of fine mirrors, each consisting of two orthogonal reflective surfaces, are arranged on a grid. The retrotransmitting optical element 31 can form an aerial image by the two orthogonal reflective surfaces reflecting light. It is generally known that a retrotransmitting optical element forms an aerial image at a position that is plane-symmetrical to the light source, with the retrotransmitting optical element as the plane of symmetry.

[0016] 1, the two mirrors 32 have a known configuration. The mirror surfaces 33 of the two mirrors 32 face each other. Hereinafter, one mirror 32 may be referred to as a first mirror 32A, and the other mirror 32 may be referred to as a second mirror 32B.

[0017] 1, the first mirror 32A, the second mirror 32B, and the retrotransmissive optical element 31 are arranged to form a triangle in a side view. Specifically, one end 32a of the first mirror 32A is connected to one end 32b of the second mirror 32B. The other end 32c of the first mirror 32A and the other end 32d of the second mirror 32B are connected to both ends of the retrotransmissive optical element 31.

[0018] In the triangle formed by the first mirror 32A, the second mirror 32B, and the retrotransmissive optical element 31, one of the three corners points downward. One of the two sides forming the one corner is formed by the retrotransmissive optical element 31 and is located on the rotating mirror 40 side. The first mirror 32A, the second mirror 32B, and the retrotransmissive optical element 31 arranged as described above are supported by a plurality of supports 23.

[0019] As shown in Fig. 1 , the user 80 is lying supine under the housing 20. For example, the head of the user 80 faces the optical system 30. The position of the head of the user 80 is generally specified by, for example, a pillow 81. The pillow 81 is disposed below the frame body 22. The user 80 visually recognizes an aerial image I3 (see Fig. 4 ) displayed by the aerial image display device 1.

[0020] Next, a mechanism by which the aerial image display device 1 displays an aerial image I3 will be described with reference to Figures 3 and 4. The aerial image I3 is formed by the light source 24, the rotating mirror 40, and the optical system 30. As shown in Figure 3, emitted light L1 from the light source 24 is incident on the rotating mirror 40. At this time, a virtual image I1 of the light source 24 is formed on the rotating mirror 40, as shown in Figure 4. As shown in Figure 3, the emitted light L1 changes direction by being reflected by the rotating mirror 40, and enters the optical system 30 as first reflected light L2.

[0021] As shown in FIG. 4 , the first reflected light L2 entering the optical system 30 first passes through the retrotransmissive optical element 31. After passing through the retrotransmissive optical element 31, the first reflected light L2 is incident on the first mirror 32A as a first refracted light L3. The first refracted light L3 is reflected by the first mirror 32A and changes direction, becoming a second reflected light L4. The second reflected light L4 is condensed and forms an aerial image I2 of the light source 24 inside the optical system 30. The second reflected light L4 again expands and is reflected by the second mirror 32B, passing through the retrotransmissive optical element 31 again as a third reflected light L5. The third reflected light L5, having passed through the retrotransmissive optical element 31, exits the optical system 30 as a second refracted light L6. The second refracted light L6 forms an aerial image I3 of the light source 24 in front of the user 80. The user 80 can view an image or video by looking into the aerial image I3.

[0022] Generally, a retrotransmitting optical system is known to image bright points close to the retrotransmitting optical element close to the retrotransmitting optical element, and bright points far from the retrotransmitting optical element far from the retrotransmitting optical element, resulting in depth inversion of the aerial image. The TRISFER optical system 30 according to this embodiment can eliminate depth inversion by combining two mirrors 32 with the retrotransmitting optical element 31. Specifically, as described above, the aerial image I3 is formed by passing through the retrotransmitting optical element 31 twice, so depth inversion occurs twice. Therefore, depth inversion is eliminated throughout the optical system 30.

[0023] As described above, the aerial image display device 1 can display the aerial image I3 in front of the user 80. The aerial image I3 displayed by the aerial image display device 1 in this manner may be movable in accordance with the rotation of the user 80's head.

[0024] The following describes a mechanism and method by which the aerial image display device 1 moves the aerial image I3 in accordance with the head rotation of the user 80. As shown in Figures 1 and 5 , the aerial image display device 1 may further include a sensor 60 and a control device 10.

[0025] The sensor 60 detects the movement of the head of the user 80. The sensor 60 may be, for example, a high-speed camera. As shown in Fig. 1 , the sensor 60 is arranged on the housing 20 so as to face the position of the head of the user 80 (for example, a pillow 81). The arrangement of the sensor 60 may be any arrangement that allows the movement of the head of the user 80 to be detected, and is not limited to the example shown in the figure.

[0026] For example, the sensor 60 may continuously take photographs of the head of the user 80 while the aerial image display device 1 is in operation. For example, the multiple photographs taken by the sensor 60 are read by the control device 10. However, the sensor 60 may have another configuration capable of detecting the movement of the head of the user 80.

[0027] As shown in Fig. 5, the control device 10 controls the light source 24 and the rotating mirror 40. As will be described later, the control device 10 receives information acquired by the sensor 60 from the sensor 60. The control device 10 is, for example, an information device such as a personal computer, a tablet, a smartphone, or a dedicated device. In the following, as an example, the control device 10 is described as being a personal computer. As shown in Fig. 6, the control device 10 includes, for example, a communication unit 11, an input unit 12, an output unit 13, a storage unit 14, and a control unit 15.

[0028] The communication unit 11 is a communication device. The communication unit 11 may be configured as, for example, a network interface. The communication unit 11 communicates data with other devices via a network in accordance with the control of the control unit 15. The communication unit 11 may be a device that performs wireless communication or a device that performs wired communication.

[0029] The input unit 12 is configured using existing input devices such as a keyboard, a pointing device (mouse, tablet, etc.), buttons, a touch panel, etc. The input unit 12 is operated by the user 80 when inputting instructions from the user 80 to the control device 10. The input unit 12 may be an interface for connecting the input device to the control device 10. In this case, the input unit 12 inputs an input signal generated in the input device in response to an input from the user 80 to the control device 10. The input unit 12 may be configured using a microphone and a voice recognition device. In this case, the input unit 12 acquires an acoustic signal generated by the speech of the user 80, voice recognizes the words spoken by the user 80, and inputs character string information of the recognition result to the control device 10. The voice recognition process may be performed by the control unit 15. The input unit 12 may be configured in any way as long as it is capable of inputting instructions from the user 80 to the control device 10.

[0030] The output unit 13 outputs information in a form recognizable by the user 80. The output unit 13 may be, for example, an image display device such as a liquid crystal display or an organic EL (Electro Luminescence) display. The output unit 13 may be an interface for connecting an image display device to the control device 10. In this case, the output unit 13 generates a video signal for displaying image data and outputs the video signal to the image display device connected to the output unit 13. The output unit 13 may be a device for outputting sound, such as a speaker. The output unit 13 may be an interface for connecting an audio output device, such as a speaker or headphones, to the control device 10. In this case, the output unit 13 generates an audio signal for reproducing audio data and outputs the audio signal to the audio output device connected to the output unit 13. The output unit 13 may be configured as a touch panel integrated with the input unit 12.

[0031] The storage unit 14 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 14 stores data used by the control unit 15. The storage unit 14 stores data required when the control unit 15 performs processing.

[0032] The control unit 15 is configured using a processor such as a CPU (Central Processing Unit) and a memory (main storage device). The processor executes a program, causing the control unit 15 to function as an image acquisition unit 151, a photo acquisition unit 152 (sensing result acquisition unit), a rotation angle estimation unit 153, a rotating mirror control unit 154, and a light source control unit 155. Note that all or part of the functions of the control unit 15 may be realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The above program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, and a semiconductor storage device (e.g., a solid-state drive (SSD)), as well as storage devices such as a hard disk or semiconductor storage device built into the computer system 100. The above program may be transmitted via a telecommunications line.

[0033] The control unit 15 may execute, for example, an application installed on its own device (the control device 10). A specific example of such an application is an application provided to the control device 10 as a dedicated application. Another specific example of such an application is a web browser application. Such an application may be pre-installed on the control device 10 or may be downloaded each time a determination process is executed. For example, when implemented as a web browser application, the control device 10 may download and execute the application from a device specified by a specific web server (for example, the web server itself or another server) in response to the control device 10 connecting to the web server. The control unit 15 operates according to the program of the application being executed.

[0034] The image acquisition unit 151 acquires an image from, for example, a terminal device different from the control device 10. The terminal device different from the control device 10 is, for example, a terminal device that outputs a VR image or the like to the HMD 25. The image acquired by the image acquisition unit 151 is an image that serves as the basis for the image visually recognized by the user 80.

[0035] The photo acquisition unit 152 acquires, for example, a plurality of photos taken by the sensor 60. The acquisition of photos by the photo acquisition unit 152 may be performed constantly, for example, while the aerial image display device 1 is in operation.

[0036] The rotation angle estimation unit 153 reads out the multiple photos acquired by the photo acquisition unit 152 and estimates the rotation angle (amount of rotation) of the head of the user 80. The rotation angle estimation unit 153 may estimate the rotation angle of the head of the user 80, for example, by comparing a photo taken at an arbitrary time point with a photo taken after a predetermined time has elapsed. In this case, for example, the rotation angle estimation unit 153 may estimate the rotation angle of the head of the user 80 by comparing the appearance of facial feature points such as the ears, nose, and mouth of the user 80. For example, the rotation angle estimation unit 153 may estimate the rotation angle of the head of the user 80 using a known algorithm, artificial intelligence, or the like. Note that the width of the predetermined time period may be stored in advance in the storage unit 14, for example.

[0037] The rotation angle of the head of the user 80 estimated by the rotation angle estimation unit 153 may be stored, for example, in the storage unit 14. The rotation angle estimation unit 153 may update the rotation angle stored in the storage unit 14 each time the head of the user 80 is rotated.

[0038] The rotating mirror control unit 154 determines the rotation angle of the rotating mirror 40 based on the rotation angle of the head of the user 80 estimated by the rotation angle estimation unit 153. Here, the relationship between the rotation of the rotating mirror 40 and the aerial image I3 displayed in front of the user 80 will be described with reference to FIGS. 7 and 8. In FIG. 7, the rotating mirror 40 after rotation is shown as a rotating mirror 40'. Furthermore, the virtual image I1 of the light source 24 reflected on the rotating mirror 40' is shown as a virtual image I1'.

[0039] As described above, the optical system 30 can eliminate depth inversion. Therefore, in the optical system 30, the trajectory of the virtual image I1 reflected on the rotating mirror 40 coincides with the trajectory of the aerial image I3 displayed in front of the user 80. For example, as shown in FIG. 7 , when the rotating mirror 40 is rotated by θ°, the virtual image I1' is projected at a position rotated by 2θ° in the direction of rotation of the rotating mirror 40. At this time, the trajectory of the virtual image I1 coincides with the trajectory of the aerial image I3, so the aerial image I3 is also projected at a position rotated by 2θ°. In other words, when the rotating mirror 40 is rotated by θ°, the aerial image I3 is displayed rotated by 2θ°.

[0040] For example, if the rotation angle of the head of the user 80 is θ°, then in order to display the aerial image I3 in accordance with the rotation of the head of the user 80, the aerial image I3 must also be rotated by θ°. As described above, in order to rotate the aerial image I3 by θ°, the rotating mirror 40 is rotated by 1 / 2θ°. In other words, by rotating the rotating mirror 40 so that the rotation angle is 1 / 2 the estimated rotation angle of the head of the user 80, the aerial image I3 can be displayed in accordance with the rotation of the head of the user 80.

[0041] As described above, the rotating mirror control unit 154 determines the rotation angle of the rotating mirror 40 as half the estimated rotation angle of the head of the user 80. The rotating mirror control unit 154 updates the signal to the motor of the rotating mirror 40 based on the determined rotation angle of the rotating mirror 40. The rotating mirror control unit 154 drives the motor based on the updated signal. This causes the rotating mirror 40 to rotate around the rotation axis O1 at a predetermined rotation angle.

[0042] As shown in Fig. 7, when the rotating mirror 40 rotates, the virtual image I1 of the light source 24 (HMD 25) moves circularly while pointing the eyepiece 26 toward the center of rotation. Therefore, as shown in Fig. 8, the aerial image I3 of the HMD 25 can be moved while always pointing the eyepiece 26 toward the user 80, and the user 80 can continue to view the image even if he or she turns his or her head without wearing the HMD 25.

[0043] The aerial image I3 displayed as described above is controlled by the light source control unit 155. The light source control unit 155 generates an image to be viewed by the user 80 based on the image acquired by the image acquisition unit 151 and the rotation angle of the user 80's head. For example, when the user 80 is viewing a VR space, an image of the VR space is generated that matches the orientation of the user 80's head. The light source control unit 155 may, for example, constantly generate an image from the image acquired by the image acquisition unit 151 in accordance with the rotation angle of the user 80's head. The generated image is transferred to the HMD 25 by, for example, the communication unit 11.

[0044] As described above, the aerial image display device 1 according to this embodiment includes the light source 24, the optical system 30 that forms an aerial image I3 of the light source 24, and the rotating mirror 40. Therefore, by rotating the rotating mirror 40, for example, it is possible to change the incident position of light incident on the optical system 30 from the light source 24, thereby changing the position where the aerial image I3 is formed. This allows the aerial image display device 1 to change the position where the aerial image I3 is formed depending on the orientation of the head of the user 80, for example. By forming the aerial image I3 directly in front of the user 80, the user 80 can view the aerial image I3 without wearing a device on their head. Therefore, the burden on the user 80 can be reduced compared to, for example, wearing a device on their head.

[0045] The aerial image display device 1 includes a control device 10. This makes it possible to automate the control of, for example, the light source 24 and the rotating mirror 40. Therefore, for example, it is possible to more accurately display the aerial image I3 in accordance with the movement of the head of the user 80.

[0046] The rotating mirror 40 rotates around the rotation axis O1 and the position of the aerial image I3 is adjusted in accordance with the amount of rotation detected by the sensor 60. This allows the aerial image display device 1 to, for example, follow the movement of the head of the user 80 and present the aerial image I3 directly in front of the user 80.

[0047] The optical system 30 includes a retrotransmissive optical element 31 including a two-layer mirror array, and two mirrors 32. This makes it possible to easily form, for example, an aerial image I3.

[0048] A control method for the aerial image display device 1 according to one aspect of the present disclosure rotates the rotating mirror 40 around the rotation axis O1 in accordance with the amount of rotation of the head of the user 80. This enables the aerial image display device 1 to follow the movement of the head of the user 80 and present an aerial image I3 directly in front of the user 80, for example.

[0049] The technical scope of the present disclosure is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present disclosure.

[0050] In the above embodiment, the optical system 30 is described as a TRISFER, but the optical system 30 is not limited to a TRISFER. For example, the optical system 30 may be configured to include a retroreflector and a beam splitter (a so-called AIRR optical system). The retroreflector and the beam splitter may be of known configurations.

[0051] In the above embodiment, the user 80 is described as lying on his / her back, but the posture of the user 80 is not limited to this. For example, the user 80 may be sitting in a reclining chair or the like.

[0052] For example, the aerial image display device 1 does not need to include the control device 10. For example, the position of the aerial image I3 may be adjusted by manually rotating the rotating mirror 40. For example, instead of the pillow 81, a mark or the like may be provided to specify the position of the head of the user 80. The pillow 81 may be omitted.

[0053] (Additional Note) The above embodiment can be understood, for example, as follows.

[0054] <1> An aerial image display device according to one aspect of the present disclosure includes a light source, an optical system that forms an aerial image of light emitted by the light source, and a rotating mirror positioned between the light source and the optical system.

[0055] An aerial image display device according to one aspect of the present disclosure includes a light source, an optical system that forms an aerial image of light emitted by the light source, and a rotating mirror. Therefore, by rotating the rotating mirror, for example, it is possible to change the incident position of light incident on the optical system from the light source, thereby changing the position where the aerial image is formed. This allows the aerial image display device to change the position where the aerial image is formed depending on the orientation of the user's head, for example. By forming the aerial image directly in front of the user, the user can view the aerial image without wearing a device on their head. Therefore, the burden on the user can be reduced compared to, for example, wearing a device on their head.

[0056] <2> The aerial image display device according to <1> above may include a control device that controls the light source and the rotating mirror.

[0057] The aerial image display device includes a control device, which can automate the control of the light source and the rotating mirror, for example, thereby enabling the aerial image to be displayed in accordance with the user's head movement with higher accuracy.

[0058] <3> The aerial image display device according to <2> above may include a sensor that detects the rotation of the user's head, and the control unit may rotate the rotating mirror around the rotation axis according to the amount of rotation of the user's head detected by the sensor.

[0059] The rotating mirror rotates around the rotation axis according to the amount of rotation detected by the sensor, and the position of the aerial image is adjusted. This allows the aerial image display device to, for example, follow the movement of the user's head and present the aerial image directly in front of the user.

[0060] <4> In the aerial image display device according to any one of the above <1> to <3>, the optical system may include a retro-transmitting optical element including a two-layer mirror array, and two mirrors.

[0061] The optical system includes a retro-transmitting optical element including a two-layer mirror array and two mirrors, which makes it possible to easily form an aerial image, for example.

[0062] <5> A control method for an aerial image display device according to one aspect of the present disclosure is a control method for an aerial image display device that includes a light source, an optical system that forms an aerial image of light emitted by the light source, and a rotating mirror positioned between the light source and the optical system, and that rotates the rotating mirror around a rotation axis in accordance with the amount of rotation of a user's head.

[0063] A method for controlling an aerial image display device according to one aspect of the present disclosure rotates a rotating mirror around a rotation axis in accordance with the amount of rotation of a user's head, thereby enabling the aerial image display device to, for example, follow the movement of the user's head and present an aerial image directly in front of the user.

[0064] According to the present disclosure, it is possible to provide an aerial image display device that can form an aerial image according to the orientation of the user's head.

[0065] REFERENCE SIGNS LIST 1 Aerial image display device 10 Control device 24 Light source 30 Optical system 31 Retrotransmitting optical element 32 Mirror 40 Rotating mirror 60 Sensor 80 User I3 Aerial image O1 Rotation axis

Claims

1. An aerial image display device comprising: a light source; an optical system that forms an aerial image of light emitted by the light source; and a rotating mirror located between the light source and the optical system.

2. The aerial image display device according to claim 1, further comprising a control device for controlling said light source and said rotating mirror.

3. An aerial image display device as described in claim 2, further comprising a sensor for detecting rotation of the user's head, and wherein the control device rotates the rotating mirror around a rotation axis in accordance with the amount of rotation of the user's head detected by the sensor.

4. An aerial image display device according to any one of claims 1 to 3, wherein the optical system includes a retro-transmitting optical element including a two-layer mirror array and two mirrors.

5. A control method for an aerial image display device comprising: a light source; an optical system that forms an aerial image of the light emitted by the light source; and a rotating mirror positioned between the light source and the optical system, wherein the rotating mirror is rotated around a rotation axis in accordance with the amount of rotation of the user's head.

Images