Airborne display device

Abstract

To provide an aerial display device which can detect an object in the vicinity of an aerial image.SOLUTION: An aerial display device includes: a display element 20 which displays an image; a reflective element 40 which is arranged so as to receive light from the display element 20 and reflects the light from the display element 20 toward the opposite side to the display element 20 to form an aerial image; and a sensing device 50 which is arranged between the display element 20 and the reflective element 40 and on the outer side of a region occupied by the reflective element 40. The sensing device 50 comprises: a light source part 52 which emits infrared light toward the reflective element 40; and a plurality of cameras 53 which detect the infrared light reflected in the order from the object and the reflective element 40.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This invention relates to an aerial display device. [Background technology]

[0002] Aerial display devices capable of displaying images and videos as aerial images are being researched and are expected to be a new human-machine interface. For example, one proposed aerial display device uses a two-sided corner reflector array, in which two-sided corner reflectors are arranged in an array, to reflect light emitted from the display surface of a display element and project a real image into the air. The display method using a two-sided corner reflector array is aberration-free, and the real image (aerial image) is displayed in a plane-symmetrical position.

[0003] If an aerial image can be manipulated with a fingertip, information can be entered without directly touching it, as with a touch panel, thus enabling hygienic input. To sense the same plane as the aerial image, sensors need to be installed near the aerial image. Furthermore, to sense along the aerial image, devices equipped with sensors need to be placed around the aerial image. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2017-67933 [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] The present invention provides an aerial display device capable of detecting objects near an aerial image. [Means for solving the problem]

[0006] According to a first aspect of the present invention, an aerial display device is provided, comprising: a display element for displaying an image; a reflecting element arranged to receive light from the display element and reflecting the light from the display element toward the opposite side of the display element to form an aerial image; and a first sensing device arranged between the display element and the reflecting element and outside the area occupied by the reflecting element, wherein the first sensing device includes a light source unit that emits infrared light toward the reflecting element and a plurality of cameras that detect the infrared light reflected sequentially by an object and the reflecting element.

[0007] According to a second aspect of the present invention, an aerial display device according to the first aspect is provided, wherein the reflective element comprises a planar substrate and a plurality of optical elements provided below the substrate, each extending in a first direction and arranged in a second direction perpendicular to the first direction, and each of the plurality of optical elements is inclined with respect to the normal direction of the substrate and has an incident surface and a reflective surface that are in contact with each other.

[0008] According to a third aspect of the present invention, an aerial display device according to the second aspect is provided, wherein the angle of the reflective surface with respect to the normal direction of the substrate is set such that the angle of incidence of light incident on the reflective surface is greater than the critical angle.

[0009] According to a fourth aspect of the present invention, an aerial display device according to the third aspect is provided, wherein the angle of the incident surface with respect to the normal direction of the substrate is set such that the angle of incidence of light incident on the incident surface is smaller than the critical angle.

[0010] According to a fifth aspect of the present invention, the first sensing device is provided as an aerial display device according to any of the second to fourth aspects, which is arranged in the second direction.

[0011] According to a sixth aspect of the present invention, an aerial display device according to any of the first to fifth aspects is provided, wherein the display element and the reflective element are arranged parallel to each other.

[0012] According to a seventh aspect of the present invention, there is provided an aerial display device according to any one of the first to sixth aspects, further comprising an alignment control element disposed between the display element and the reflection element and transmitting a part of the light from the display element.

[0013] According to an eighth aspect of the present invention, there is provided an aerial display device according to any one of the first to seventh aspects, further comprising a housing that houses the display element, the reflection element, and the first sensing device.

[0014] According to a ninth aspect of the present invention, there is provided an aerial display device according to any one of the first to eighth aspects, further comprising a control unit that calculates the position of a detected object based on the image signals of the plurality of cameras.

[0015] According to a tenth aspect of the present invention, there is provided an aerial display device according to any one of the first to ninth aspects, further comprising a second sensing device that is disposed between the display element and the reflection element and outside the region occupied by the reflection element, is disposed on the opposite side of the reflection element from the first sensing device, and has the same configuration as the first sensing device.

Advantages of the Invention

[0016] According to the present invention, it is possible to provide an aerial display device capable of detecting an object near an aerial image.

Brief Description of the Drawings

[0017] [Figure 1] FIG. 1 is a perspective view of an aerial display device according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a perspective view of the display module and the sensing device shown in FIG. 1. [Figure 3] FIG. 3 is a side view of the display module and the sensing device shown in FIG. 1 as viewed from the Y direction. [Figure 4A] FIG. 4A is a plan view of the alignment control element shown in FIG. 2. [Figure 4B] FIG. 4B is a cross-sectional view of the alignment control element taken along line A-A' of FIG. 4A. [Figure 5] Figure 5 shows a perspective view and a partially enlarged view of the reflective element shown in Figure 2. [Figure 6] Figure 6 is a schematic plan view of the sensing device shown in Figure 1. [Figure 7] Figure 7 is a block diagram of the aerial display device. [Figure 8] Figure 8 is a perspective view illustrating how light is reflected by a reflective element. [Figure 9] Figure 9 is a side view of the XZ plane illustrating the reflection of light in a reflective element. [Figure 10] Figure 10 is a side view of the YZ plane illustrating the reflection of light in a reflective element. [Figure 11] Figure 11 illustrates the angular conditions of the incident and reflective surfaces in a reflective element. [Figure 12] Figure 12 is a ray tracing diagram of an aerial display device. [Figure 13] Figure 13 is a diagram illustrating the operation of the sensing device. [Figure 14] Figure 14 is a ray tracing diagram of the sensing device. [Figure 15] Figure 15 is a perspective view of an aerial display device according to a second embodiment of the present invention. [Figure 16] Figure 16 is a side view of an aerial display device according to a third embodiment of the present invention. [Figure 17] Figure 17 is a perspective view of an aerial display device according to a fourth embodiment of the present invention. [Figure 18] Figure 18 is a side view of the display module and sensing device shown in Figure 17, as seen from the Y direction. [Figure 19A] Figure 19A is a side view of the reflective element shown in Figure 18, as seen from the Y direction. [Figure 19B] Figure 19B is a bottom view of the reflective element shown in Figure 18. [Figure 20A] Figure 20A is a plan view of the optical path deflection element shown in Figure 18. [Figure 20B]Figure 20B is a side view of the optical path deflection element shown in Figure 18, viewed from the Y direction. [Figure 21] Figure 21 is a perspective view of an aerial display device according to a modified example of the fifth embodiment of the present invention. [Modes for carrying out the invention]

[0018] The embodiments will be described below with reference to the drawings. However, the drawings are schematic or conceptual, and the dimensions and proportions in each drawing are not necessarily the same as those in reality. Furthermore, even when the same part is represented between drawings, the relationship between dimensions and proportions may be represented differently. In particular, the embodiments shown below are illustrative examples of devices and methods for realizing the technical concept of the present invention, and the technical concept of the present invention is not determined by the shape, structure, arrangement, etc. of the components. In the following description, elements having the same function and configuration are denoted by the same reference numerals, and redundant descriptions are omitted.

[0019] [1] First Embodiment [1-1] Configuration of the aerial display device 1 Figure 1 is a perspective view of an aerial display device 1 according to a first embodiment of the present invention. In Figure 1, the X direction is the direction along one side of the aerial display device 1, the Y direction is the direction perpendicular to the X direction in the horizontal plane, and the Z direction is the direction perpendicular to the XY plane (also called the normal direction). The aerial display device 1 comprises a housing 2, a display module 3, and a sensing device 50.

[0020] The housing 2 houses the display module 3 and the sensing device 50. That is, the housing 2, the display module 3, and the sensing device 50 are integrally constructed. The housing 2 has a rectangular parallelepiped shape and also a box shape. The housing 2 includes support members (not shown) for fixing the display module 3 and the sensing device 50 in the illustrated positions. The housing 2 has an opening 2A that exposes the upper surface (light-emitting surface) of the display module 3. The housing 2 is made of metal or resin.

[0021] Display module 3 is a device that displays images (including videos). Display module 3 displays an aerial image in the air above its light-emitting surface. An aerial image is a real image formed in the air. Display module 3 includes multiple elements stacked at intervals in the Z direction. Note that in Figure 1, for simplification, only some of the multiple elements included in display module 3 are shown. Details of display module 3 will be described later.

[0022] The sensing device 50 is a device for detecting objects included in a predetermined area near an aerial image. The sensing device 50 has the shape of a rectangular parallelepiped. The sensing device 50 emits light in a predetermined direction. The sensing device 50 also detects light reflected by the object. Details of the sensing device 50 will be described later.

[0023] Figure 2 is a perspective view of the display module 3 and sensing device 50 shown in Figure 1. Figure 3 is a side view of the display module 3 and sensing device 50 shown in Figure 1, viewed from the Y direction.

[0024] The display module 3 comprises an illumination element (also called a backlight) 10, a display element 20, an orientation control element 30, and a reflective element 40. The illumination element 10, the display element 20, the orientation control element 30, and the reflective element 40 are arranged parallel to each other. The illumination element 10, the display element 20, the orientation control element 30, and the reflective element 40 are fixed in the position shown in Figure 2 by a support member (not shown) provided inside the housing 2.

[0025] The illumination element 10 emits illumination light and directs this illumination light toward the display element 20. The illumination element 10 comprises a light source unit 11, a light guide plate 12, and a reflective sheet 13. The illumination element 10 is, for example, a side-light type illumination element. The illumination element 10 constitutes a surface light source. The illumination element 10 may be configured so that the light intensity peaks in an oblique direction at an angle θ1, as described later.

[0026] The light source unit 11 emits light. The light source unit 11 includes a plurality of light-emitting elements, such as white LEDs (Light Emitting Diodes). The light guide plate 12 guides the illumination light from the light source unit 11 and emits the illumination light from its top surface. The reflective sheet 13 reflects the illumination light emitted from the bottom surface of the light guide plate 12 back towards the light guide plate 12. The illumination element 10 may be equipped with optical elements (including a prism sheet and a diffusion sheet) on the top surface of the light guide plate 12 to improve its optical properties.

[0027] The display element 20 is a transmissive display element. The display element 20 is composed of, for example, a liquid crystal display element. The driving mode of the display element 20 is not particularly limited, and TN (Twisted Nematic) mode, VA (Vertical Alignment) mode, or homogeneous mode can be used. The display element 20 receives illumination light emitted from the illumination element 10. The display element 20 transmits the illumination light and performs light modulation. Then, the display element 20 displays a desired image on its display surface.

[0028] The orientation control element 30 has the function of reducing unwanted light. Unwanted light is a light component that does not contribute to the generation of an aerial image and is a light component that is transmitted through the reflecting element 40 in the normal direction. The orientation control element 30 is configured to block light components outside a predetermined angular range centered on an oblique direction at an angle θ1 with respect to the normal direction.

[0029] The reflecting element 40 reflects light incident from the bottom side to the top side. It also reflects incident light that enters obliquely from the bottom side, for example, in the forward direction (normal direction). The reflecting element 40 then forms an aerial image 4 in the air. An observer 5 standing directly in front of the reflecting element 40 can see the aerial image 4.

[0030] [1-1-1] Configuration of the orientation control element 30 Figure 4A is a plan view of the orientation control element 30 shown in Figure 2. Figure 4B is a cross-sectional view of the orientation control element 30 along line AA' in Figure 4A.

[0031] The base materials 31 and 32 are each configured in a rectangular planar shape. Multiple transparent members 33 are provided on base material 31, each extending in the Y direction and aligned in the X direction. Multiple light-shielding members 34 are also provided on base material 31, each extending in the Y direction and aligned in the X direction. The multiple transparent members 33 and multiple light-shielding members 34 are arranged alternately so that adjacent ones are in contact with each other. Base material 32 is provided on the multiple transparent members 33 and multiple light-shielding members 34.

[0032] The transparent member 33 extends in an oblique direction at an angle θ1 with respect to the normal direction of the base material 31 in the X direction. In the XZ cross section, the transparent member 33 is a parallelogram with its side surface inclined by an angle θ1. The transparent member 33 transmits light.

[0033] The light-shielding member 34 extends in an oblique direction at an angle θ1 with respect to the normal direction of the base material 31 in the X direction. In the XZ cross section, the light-shielding member 34 is a parallelogram with its side surface inclined by an angle θ1. The light-shielding member 34 blocks light.

[0034] The two adjacent light-shielding members 34 are arranged so that their ends slightly overlap in the Z direction.

[0035] A transparent resin is used for the base materials 31, 32, and the transparent member 33, for example, acrylic resin. For the light-shielding member 34, for example, a resin mixed with black dye is used.

[0036] The orientation control element 30 configured in this way can transmit display light such that the light intensity in the oblique direction at an angle θ1 with respect to the normal direction is at its peak. For example, the orientation control element 30 is configured to block light components outside the range of 30°±30° with respect to the normal direction. Preferably, the orientation control element 30 is configured to block light components outside the range of 30°±20° with respect to the normal direction.

[0037] As a variation, the orientation control element 30 may be placed between the illumination element 10 and the display element 20. Alternatively, the orientation control element 30 may be placed closer to the reflective element 40. Furthermore, the aerial display device 1 may be configured without the orientation control element 30.

[0038] [1-1-2] Configuration of the reflector element 40 Figure 5 shows a perspective view and a partially enlarged view of the reflective element 40 shown in Figure 2. The partially enlarged view in Figure 5 is an enlarged view of the XZ plane.

[0039] The reflective element 40 comprises a base material 41 and a plurality of optical elements 42. The base material 41 is planar in the XY plane and has a rectangular parallelepiped shape.

[0040] Multiple optical elements 42 are provided on the bottom surface of the base material 41. Each of the multiple optical elements 42 is composed of a triangular prism. The optical elements 42 are arranged such that three sides of the triangular prism are parallel to the XY plane, and one side is in contact with the base material 41. Each of the multiple optical elements 42 extends in the Y direction and is arranged in a line in the X direction. In other words, the multiple optical elements 42 have a sawtooth shape.

[0041] The optical element 42 has an incident surface 43 and a reflective surface 44. The incident surface 43 is the surface to which light from the orientation control element 30 is incident. The reflective surface 44 is the surface that reflects light incident on the incident surface 43 from the outside within the optical element 42.

[0042] The optical element 42 is formed integrally with the substrate 41, for example, from the same transparent material as the substrate 41. Alternatively, the substrate 41 and the optical element 42 may be formed separately, and the optical element 42 may be bonded to the substrate 41. As the transparent material constituting the substrate 41 and the optical element 42, for example, acrylic resin or glass can be used.

[0043] The reflective element 40 reflects incident light internally, forming a real image in the air. The reflective element 40 also forms the real image directly in front of its element surface. The element surface refers to the imaginary plane on which the reflective element 40 extends. The same meaning applies to the element surfaces of other elements.

[0044] [1-1-3] Configuration of the sensing device 50 Figure 6 is a schematic plan view of the sensing device 50 shown in Figure 1. The sensing device 50 comprises a housing 51, a light source unit 52, and two cameras 53-1 and 53-2. The number of cameras 53 may be three or more.

[0045] The housing 51 houses the light source unit 52 and cameras 53-1 and 53-2. The housing 51 consists of a box-shaped case and a cover that covers the top of the case. The cover of the housing 51 has openings that expose the light source unit and the cameras.

[0046] The light source unit 52 emits infrared light perpendicular to the element surface of the sensing device 50. It is desirable that the light emitted by the light source unit 52 has a wavelength that does not affect the image displayed by the display element 20. The light source unit 52 includes one or more LEDs.

[0047] Cameras 53-1 and 53-2 detect infrared light reflected from an object. Cameras 53-1 and 53-2 are composed of infrared cameras and include an image sensor. The image sensor is composed of a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. Cameras 53-1 and 53-2 constitute a stereo camera. A stereo camera can acquire three-dimensional position information by using multiple cameras (for example, two cameras) to image an object from multiple directions. In other words, a stereo camera can acquire three-dimensional position information by adding depth information to two-dimensional position information parallel to an aerial image.

[0048] As shown in Figure 2, the sensing device 50 is positioned outside the area occupied by the reflecting element 40 in a plan view, and is positioned between the display element 20 and the reflecting element 40. The sensing device 50 is positioned on the side of the reflecting element 40 that is aligned with the X direction, facing the incident surface 43 of the reflecting element 40. The sensing device 50 is positioned so that its element surface faces diagonally upward so that the emitted light irradiates the reflecting element 40. The sensing device 50 is positioned at an angle of, for example, 10 degrees to 60 degrees.

[0049] [1-2] Block configuration of the aerial display device 1 Figure 7 is a block diagram of the aerial display device 1. The aerial display device 1 comprises a control unit 60, a storage unit 61, an input / output interface (input / output IF) 62, a display module 3, and an input unit 63. The control unit 60, the storage unit 61, and the input / output interface 62 are connected to each other via a bus.

[0050] The input / output interface 62 is connected to the display module 3, the sensing device 50, and the input unit 63. The input / output interface 62 performs interface processing for each of the display module 3, the sensing device 50, and the input unit 63 according to a predetermined standard.

[0051] The display module 3 includes an illumination element 10 and a display element 20.

[0052] The sensing device 50 comprises a light source unit 52, a plurality of cameras 53, and a sensor controller 54. The sensor controller 54 controls the operation of the light source unit 52 and the plurality of cameras 53 based on the control of the control unit 60. The sensor controller 54 controls the light emission timing of the light source unit 52. The sensor controller 54 receives image signals from the plurality of cameras 53.

[0053] The control unit 60 is composed of one or more processors, such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 60 implements various functions by executing programs stored in the storage unit 61. The control unit 60 includes a display processing unit 60A, a position calculation unit 60B, and an information processing unit 60C.

[0054] The display processing unit 60A controls the operation of the display module 3 (specifically, the illumination element 10 and the display element 20). The display processing unit 60A transmits an image signal to the display module 3, causing the display module 3 to display an image.

[0055] The position calculation unit 60B controls the operation of the sensing device 50. The position calculation unit 60B receives image signals from the sensing device 50. The position calculation unit 60B causes the sensing device 50 to form a detection area for detecting objects using infrared light. Based on the image signals from the multiple cameras 53, the position calculation unit 60B calculates the position of objects that have entered the detection area. The distance to the object is calculated using the principle of triangulation, a known technique. The detected position includes the position where the user touches the aerial image with their finger.

[0056] The information processing unit 60C generates the image to be displayed by the aerial display device 1. The information processing unit 60C can use the image data stored in the storage unit 61. The information processing unit 60C uses a communication function (not shown) to communicate externally. from You may obtain image data.

[0057] The storage unit 61 includes non-volatile storage devices such as ROM (Read Only Memory), HDD (Hard Disk Drive), and SSD (Solid State Drive), and volatile storage devices such as RAM (Random Access Memory) and registers. The storage unit 61 stores the program executed by the control unit 60. The storage unit 61 stores various data necessary for controlling the control unit 60. The storage unit 61 stores the image data displayed by the aerial display device 1.

[0058] The input unit 63 receives information entered by the user. The information processing unit 60C can select an image based on the information received by the input unit 63.

[0059] [1-3] Operation of the aerial display device 1 Next, the operation of the aerial display device 1 configured as described above will be explained.

[0060] [1-3-1] Display operation The arrows in Figure 3 indicate the optical path. As shown in Figure 3, light emitted from the display element 20 enters the orientation control element 30. Of the light emitted from the display element 20, the optical component at angle θ1 (including the optical component within a predetermined angular range centered on angle θ1) passes through the orientation control element 30. The light that has passed through the orientation control element 30 enters the reflecting element 40. The reflecting element 40 images the incident light into the air on the opposite side of the orientation control element 30, displaying an aerial image 4 in the air.

[0061] Figure 8 is a perspective view illustrating the reflection of light in the reflecting element 40. Figure 9 is a side view of the XZ plane illustrating the reflection of light in the reflecting element 40. Figure 9 shows the reflecting element 40 as seen by observer 5 with both eyes (i.e., the line connecting both eyes) parallel to the X direction. Figure 10 is a side view of the YZ plane illustrating the reflection of light in the reflecting element 40. Figure 10 shows the reflecting element 40 as seen by observer 5 with both eyes parallel to the Y direction.

[0062] Light emitted from an arbitrary point "o" on the element surface of the orientation control element 30 enters the reflecting element 40 from the incident surface 43 and reaches the reflecting surface 44. Light arriving at an angle greater than the critical angle with respect to the normal direction of the reflecting surface 44 is totally reflected by the reflecting surface 44 and is emitted from the plane opposite to the side of the reflecting element 40 where the optical element 42 is formed. The critical angle is the smallest incident angle beyond which total reflection occurs. The critical angle is the angle with respect to the perpendicular to the incident surface.

[0063] In the XZ plane of Figure 9, light emitted from point "o" is totally reflected by the reflective surface 44 of the optical element 42, and the light is imaged in the air to generate an aerial image.

[0064] In the YZ plane of Figure 10, the light emitted from point "o" is not reflected by the reflective surface 44 of the optical element 42, and therefore does not form an image in the air, thus not contributing to the generation of an aerial image.

[0065] In other words, the condition for observer 5 to perceive the aerial image is that both of observer 5's eyes are parallel to or close to the X direction (for example, ±10 degrees to the X direction). Furthermore, if observer 5 moves their viewpoint along the Y direction while both of their eyes are parallel to or close to the X direction, they can always perceive the aerial image.

[0066] Figure 11 illustrates the angular conditions of the incident surface 43 and the reflective surface 44 in the reflecting element 40.

[0067] The angle of the incident surface 43 with respect to the Z direction (direction perpendicular to the element surface) is θ2, the angle of the reflecting surface 44 with respect to the Z direction is θ3, and the angle between the incident surface 43 and the reflecting surface 44 is θ p Let's assume that. angle θ p This can be expressed by the following equation (1). θ p =θ² + θ³···(1) Light emitted from the orientation control element 30 at an angle θ1 is incident on the incident surface 43. The refractive index of the material of the reflecting element 40 is n p Assume the refractive index of air is 1. Let the angle of incidence at the incident surface 43 be θ4 and the angle of refraction be θ5. Let the angle of incidence at the reflecting surface 44 be θ6 and the angle of reflection be θ7 (=θ6). Let the angle of incidence at the upper surface of the orientation control element 30 be θ8 and the angle of refraction be θ9. The angle of refraction θ9 is the exit angle. The exit angle θ9 is expressed by the following equation (2). θ9 = sin -1 (n p *sin(sin -1 ((1 / n p )*sin(90°-(θ1+θ2)))+θ2+2θ3-90°)) ···(2) The critical angle on the reflecting surface 44 is represented by the following formula (3). Critical angle <θ6(=θ7) Critical angle = sin -1 (1 / n p ) ···(3) That is, the incident angle θ6 on the reflecting surface 44 is set to be larger than the critical angle on the reflecting surface 44. In other words, the angle θ3 of the reflecting surface 44 is set so that the incident angle of the light incident on the reflecting surface 44 is larger than the critical angle.

[0068] Also, the light incident on the incident surface 43 is set not to be totally reflected by the incident surface 43. That is, the angle θ2 of the incident surface 43 is set so that the incident angle of the light incident on the incident surface 43 is smaller than the critical angle.

[0069] FIG. 12 is a ray tracing diagram of the aerial display device 1. The parameters in FIG. 12 are θ1 = 35 degrees, θ3 = 22.5 degrees, θ p = 45 degrees. The light emitted from the orientation control element 30 is reflected by the reflection element 40 to form the aerial image 4.

[0070] [1-3-2] Sensing operation Next, the sensing operation will be described. FIG. 13 is a diagram for explaining the operation of the sensing device 50.

[0071] The light source unit 52 of the sensing device 50 emits infrared light toward the reflection element 40. The light source unit 52 emits infrared light with a predetermined divergence angle. The divergence angle is the angle with respect to the direction perpendicular to the element surface of the sensing device 50. The light source unit 52 is configured to be able to radiate infrared light over the entire surface of the reflection element 40. The light source unit 52 may have a lens for controlling the radiation area.

[0072] The infrared light emitted from the light source unit 52 enters from the reflecting surface 44 of the reflection element 40 and is totally reflected by the incident surface 43. The infrared light reflected by the reflection element 40 is emitted on the side of the reflection element 40 opposite to the sensing device 50 and approximately in the normal direction. Thereby, a detection area 55 by infrared light is formed.

[0073] In Figure 13, a finger is used as an example of the object 6. Infrared light emitted from the reflecting element 40 is reflected by the object 6. The light reflected by the object 6 follows the opposite optical path to the infrared light emitted from the light source unit 52 and undergoes total internal reflection at the incident surface 43 of the reflecting element 40. The infrared light reflected by the reflecting element 40 is received and detected by multiple cameras 53 of the sensing device 50.

[0074] Image signals captured by multiple cameras 53 are transmitted to the control unit 60. The position calculation unit 60B calculates the position of the object 6 that has entered the detection area 55 based on the image signals from the multiple cameras 53.

[0075] Figure 14 is a ray tracing diagram of the sensing device 50. The infrared light emitted from the light source unit 52 of the sensing device 50 illuminates almost the entire surface of the reflecting element 40.

[0076] The infrared light reflected by the reflecting element 40 has a higher light density as it moves to the right (closer to the sensing device 50) and a lower light density as it moves to the right (farther from the sensing device 50). The infrared light intensity may also be defined as the photon flux density.

[0077] [1-4] Effects of the first embodiment According to the first embodiment, an aerial display device 1 capable of detecting objects near the aerial image 4 can be realized.

[0078] Furthermore, by utilizing the reflected light from the reflective element 40, a wide detection area 55 can be formed with a single sensing device 50. In addition, the object 6 can be detected in a wide area including the aerial image 4.

[0079] Furthermore, by using multiple cameras 53 to image the object 6, three-dimensional positional information can be obtained.

[0080] Furthermore, the sensing device 50 can be housed in the housing 2. This allows the aerial display device 1, including the sensing device 50, to be configured as a single unit, and also allows the aerial display device 1 to be miniaturized.

[0081] Furthermore, it is possible to realize an aerial display device 1 that can display aerial images while ensuring display quality.

[0082] Furthermore, when observer 5 views the reflecting element 40 with both eyes parallel to or nearly parallel to the X direction (i.e., the direction in which the multiple optical elements 42 are aligned), observer 5 can perceive the aerial image. Also, when observer 5 moves their viewpoint along the Y direction with both eyes parallel to or nearly parallel to the X direction, they can always perceive the aerial image. In other words, a field of view can be secured when observer 5's eyes are parallel to or nearly parallel to the X direction.

[0083] Furthermore, multiple elements constituting the aerial display device 1 can be arranged in parallel. This makes it possible to realize an aerial display device 1 that can be miniaturized in the Z direction.

[0084] Furthermore, an aerial image can be displayed in the front direction of the aerial display device 1.

[0085] [2] Second embodiment The second embodiment is a configuration example in which the aerial display device 1 is equipped with two sensing devices 50.

[0086] Figure 15 is a perspective view of the aerial display device 1 according to a second embodiment of the present invention.

[0087] The aerial display device 1 comprises two sensing devices 50-1 and 50-2. The configuration of each of the sensing devices 50-1 and 50-2 is the same as that of the sensing device 50 described in the first embodiment.

[0088] The sensing device 50-1 is positioned outside the area occupied by the reflecting element 40 in a plan view, and is positioned between the display element 20 and the reflecting element 40. The sensing device 50-1 is positioned to the right of the reflecting element 40 in the X direction, facing the incident surface 43 of the reflecting element 40. The sensing device 50-1 is positioned so that its element surface faces diagonally upward so that the emitted light irradiates the reflecting element 40. The sensing device 50-1 is positioned at an angle of, for example, 10 degrees or more and 60 degrees or less.

[0089] The sensing device 50-2 is positioned outside the area occupied by the reflecting element 40 in a plan view, and is positioned between the display element 20 and the reflecting element 40. The sensing device 50-2 is positioned to the left of the reflecting element 40 in the X direction, facing the reflective surface 44 of the reflecting element 40. The sensing device 50-2 is positioned so that its element surface faces diagonally upward so that the emitted light irradiates the reflecting element 40. The sensing device 50-2 is positioned at an angle of, for example, 10 degrees or more and 60 degrees or less.

[0090] The position calculation unit 60B calculates the position of an object that has entered the detection area based on the image signals from the sensing devices 50-1 and 50-2.

[0091] According to the second embodiment, the intensity of infrared light emitted by sensing devices 50-1 and 50-2 can be made more uniform in the X direction. This improves the sensitivity of sensing device 50. Other effects are the same as in the first embodiment.

[0092] [3] Third embodiment The third embodiment is a configuration example in which the display element 20 is positioned diagonally with respect to the reflective element 40.

[0093] Figure 16 is a side view of an aerial display device 1 according to a third embodiment of the present invention. In Figure 16, the Z direction is the direction along one side of the display element 20, the X direction is the direction perpendicular to the Z direction, and the Y direction is the depth direction of the drawing.

[0094] The aerial display device 1 comprises an illumination element 10, a display element 20, and a reflecting element 40. The configurations of each of the illumination element 10, the display element 20, and the reflecting element 40 are the same as in the first embodiment.

[0095] The reflective element 40 is at an angle θ with respect to the display element 20. 10 They are positioned diagonally at an angle θ. 10 For example, it is between 30 and 60 degrees.

[0096] The display element 20 emits light perpendicular to its surface. The light emitted from the display element 20 is incident on the reflecting element 40. The reflecting element 40 reflects light incident at an oblique angle from the bottom side to the top side. For example, the reflecting element 40 forms an aerial image 4 of the image displayed on the display element 20 at a position symmetrical with respect to the reflecting element 40 as the plane of symmetry. The angle θ is relative to the normal direction of the reflecting element 40. 10 Observer 5, positioned at an angle, can see the aerial image 4.

[0097] Furthermore, the image formation position of the aerial image 4 can be arbitrarily set according to the angle between the display element 20 and the reflecting element 40, and the angle of the surface of the optical element 42 of the reflecting element 40.

[0098] The sensing device 50 is positioned between the display element 20 and the reflecting element 40, and outside the area occupied by the reflecting element 40 when viewed from the Z direction. The sensing device 50 is positioned so that its element surface faces the bottom surface of the reflecting element 40. Furthermore, the sensing device 50 is positioned so that the optical axis of the light source unit 52 passes near the center of the bottom surface of the reflecting element 40. The configuration and operation of the sensing device 50 are the same as in the first embodiment.

[0099] According to the third embodiment, the aerial display device 1 can be realized by arranging the display element 20 diagonally with respect to the reflective element 40. Other effects are the same as in the first embodiment.

[0100] [4] Fourth Embodiment The fourth embodiment is another configuration example of the reflective element 40, in which the reflective element 40 is composed of a two-sided corner reflector array.

[0101] [4-1] Configuration of the aerial display device 1 Figure 17 is a perspective view of an aerial display device 1 according to a fourth embodiment of the present invention. The aerial display device 1 comprises a housing 2, a display module 3, and four sensing devices 50-1 to 50-4. In Figure 17, for simplification, only some of the elements included in the display module 3 are shown.

[0102] Sensing device 50-1 is positioned on one end of the display module 3 in the X direction, and sensing device 50-3 is positioned on the other end of the display module 3 in the X direction. Sensing device 50-2 is positioned on one end of the display module 3 in the Y direction, and sensing device 50-4 is positioned on the other end of the display module 3 in the Y direction. The configuration of each of the sensing devices 50-1 to 50-4 is the same as that of sensing device 50 described in the first embodiment.

[0103] Figure 18 is a side view of the display module 3 and sensing device 50 shown in Figure 17, viewed from the Y direction. In Figure 18, the sensing devices 50-2 and 50-4 are not shown.

[0104] The display module 3 comprises an illumination element 10, a display element 20, an orientation control element 30, a reflector element 40, and an optical path deflection element 70. The configuration of the illumination element 10, the display element 20, and the orientation control element 30 is the same as in the first embodiment.

[0105] The reflective element 40 reflects light incident from the bottom side to the top side. Furthermore, the reflective element 40 reflects incident light symmetrically with respect to its surface. The reflective element 40 then forms an aerial image.

[0106] The optical path deflection element 70 refracts the display light, which is incident at an angle from the bottom surface, in the direction of the normal. An observer 5 standing in front of the optical path deflection element 70 can see the aerial image 4.

[0107] [4-2] Configuration of the reflector element 40 Figure 19A is a side view of the reflective element 40 shown in Figure 18, viewed from the Y direction. Figure 19B is a bottom view of the reflective element 40 shown in Figure 18.

[0108] The reflective element 40 comprises a base material 41 and a plurality of optical elements 45. Figure 19A shows a perspective view of one optical element 45. The base material 41 and the plurality of optical elements 45 are made of a transparent material such as acrylic resin. The base material 41 and the plurality of optical elements 45 may be formed as a single unit or bonded together with a transparent adhesive.

[0109] Multiple optical elements 45 are provided on the bottom surface of the substrate 41. The optical elements 45 consist of a rectangular parallelepiped or a cube. The planar shape of the optical elements 45 is, for example, a square. The optical elements 45 have two reflective surfaces 46 and 47. The reflective surfaces 46 and 47 correspond to the two sides of the rectangular parallelepiped and are in contact with each other. The reflective surfaces 46 and 47 constitute a so-called two-sided corner reflector.

[0110] The optical element 45 has one side at an angle θ with respect to the X direction. 11 They are positioned so that they are tilted by only θ. 11 For example, this is 45 degrees. 11 teeth, The angle is not limited to 45 degrees, but can be set within a range of 30 to 60 degrees. The multiple optical elements 45 are arranged in a staggered pattern. That is, the multiple optical elements 45 are arranged such that one row extends in a direction 45 degrees to the X direction, and multiple rows are aligned in a direction 45 degrees to the Y direction. In addition, the multiple optical elements 45 are arranged with gaps between them.

[0111] Light that has passed through the orientation control element 30 enters the optical element 45 from its bottom surface, is reflected twice by the reflective surfaces 46 and 47, and then exits from the top surface of the optical element 45.

[0112] [4-3] Configuration of the optical path deflection element 70 Figure 20A is a plan view of the optical path deflection element 70 shown in Figure 18. Figure 20B is a side view of the optical path deflection element 70 shown in Figure 18, viewed from the Y direction.

[0113] The optical path deflection element 70 comprises a base material 71 and a plurality of triangular prisms 72 provided on the base material 71. The base material 71 and the plurality of triangular prisms 72 are made of a transparent material such as acrylic resin. The base material 71 and the plurality of triangular prisms 72 may be formed as a single unit or they may be bonded together with a transparent adhesive.

[0114] Multiple triangular prisms 72 each extend in the Y direction and are aligned in the X direction. Each triangular prism 72 consists of a triangular prism extending in the Y direction. Each triangular prism 72 has a refractive surface 73. The refractive surface 73 is at an angle θ with respect to the normal direction. 12 It's only tilted.

[0115] The optical path deflection element 70 configured in this way refracts the display light incident at an angle from the bottom surface in the direction of the normal. The angle θ of the refractive surface 73 12 This is set appropriately according to the angle of light incident on the optical path deflection element 70 and the refractive index of the optical path deflection element 70.

[0116] The optical path deflection element 70 may be omitted. In this case, the aerial image is formed at a position symmetrical to the reflecting element 40.

[0117] [4-4] Sensing operation In the aerial display device 1 configured as described above, the operation of each of the sensing devices 50-1 to 50-4 is the same as that of the sensing device 50 described in the first embodiment.

[0118] Image signals captured by multiple cameras 53 included in sensing devices 50-1 to 50-4 are transmitted to the control unit 60. The position calculation unit 60B calculates the position of the object 6 that has entered the detection area 55 based on the image signals from the multiple cameras 53.

[0119] [4-5] Effects of the fourth embodiment According to the fourth embodiment, even when the reflective element 40 is configured with a two-sided corner reflector array, the object can be detected using the sensing device 50.

[0120] In the fourth embodiment, there are no restrictions on the position where the sensing device 50 is placed. In the fourth embodiment, four sensing devices 50-1 to 50-4 are shown, but the number of sensing devices 50 may be one or two or more.

[0121] Furthermore, the third embodiment may be applied to the fourth embodiment. That is, the display element 20 may be positioned diagonally with respect to the reflective element 40.

[0122] [5] Fifth embodiment The fifth embodiment is a modification of the fourth embodiment.

[0123] Figure 21 is a perspective view of an aerial display device 1 according to a modified example of the fifth embodiment of the present invention. The aerial display device 1 comprises four sensing devices 50-1 to 50-4.

[0124] Sensing devices 50-1 to 50-4 are positioned near the four corners of the display module 3. The rest of the configuration is the same as in the fourth embodiment.

[0125] In the fifth embodiment as well, the sensing device 50 can be used to detect the target object.

[0126] In the embodiments described above, a liquid crystal display device is used as an example of the display element 20, but the invention is not limited to this. The display element 20 can also be a self-emissive organic EL (electroluminescence) display element or a micro-LED (light-emitting diode) display element. A micro-LED display element is a display element that emits R (red), G (green), and B (blue) light, which constitute the pixels, using LEDs.

[0127] The present invention is not limited to the embodiments described above, and can be modified in various ways during implementation without departing from its essence. Furthermore, each embodiment may be combined as appropriate, and in that case, the combined effects can be obtained. Moreover, the above embodiments include various inventions, and various inventions can be extracted by selecting combinations from the multiple constituent elements disclosed. For example, if the problem can be solved and effects obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiment, then the configuration with these deleted constituent elements can be extracted as an invention. [Explanation of symbols]

[0128] 1...Aerial display device, 2...Housing, 2A...Aperture, 3...Display module, 4...Aerial image, 5...Observer, 6...Object, 10...Illumination element, 11...Light source unit, 12...Light guide plate, 13...Reflective sheet, 20...Display element, 30...Orientation control element, 31,32...Substrate, 33...Transparent member, 34...Light shielding member, 40...Reflective element, 41...Substrate, 42,45...Optical elements, 50...Sensing device, 51...Housing, 52...Light source unit, 53...Camera, 54...Sensor controller, 55...Detection area, 60...Control unit, 61...Storage unit, 62...Input / output interface, 63...Input unit, 70...Optical path deflection element, 71...Substrate, 72...Triangular prism.

Claims

1. A display element that displays an image, A reflecting element is positioned to receive light from the display element and reflects the light from the display element toward the opposite side of the display element to form an aerial image, A first sensing device is positioned on the same side as the display element relative to the reflective element, It is equipped with, The first sensing device is A light source unit that emits infrared light toward the reflecting element, The system includes a plurality of cameras that detect infrared light reflected sequentially from the object and the reflecting element, The reflective element comprises a planar substrate and a plurality of optical elements provided beneath the substrate, each extending in a first direction and arranged in a second direction perpendicular to the first direction. Each of the plurality of optical elements has an incident surface and a reflective surface that are inclined with respect to the normal direction of the substrate and are in contact with each other. The first sensing device is configured to emit infrared light parallel to the second direction. Aerial display device.

2. The angle of the reflective surface with respect to the normal direction of the substrate is set such that the angle of incidence of light incident on the reflective surface is greater than the critical angle. The aerial display device according to claim 1.

3. The angle of the incident surface with respect to the normal direction of the substrate is set such that the angle of incidence of light incident on the incident surface is smaller than the critical angle. The aerial display device according to claim 1 or 2.

4. The display element and the reflective element are arranged parallel to each other. The aerial display device according to any one of claims 1 to 3.

5. The system further comprises an orientation control element disposed between the display element and the reflecting element, which transmits a portion of the light from the display element. The aerial display device according to any one of claims 1 to 4.

6. The device further comprises a housing that accommodates the display element, the reflective element, and the first sensing device. The aerial display device according to any one of claims 1 to 5.

7. The system further comprises a control unit that calculates the position of a detected object based on the image signals from the multiple cameras. The aerial display device according to any one of claims 1 to 6.

8. The system further comprises a second sensing device which is positioned on the same side as the display element with respect to the reflective element and on the opposite side of the first sensing device with respect to the reflective element and which has the same configuration as the first sensing device. The aerial display device according to any one of claims 1 to 7.

Images