Airborne display device

Abstract

To provide an aerial display device capable of accurately detecting a touch operation of an observer to an aerial image.SOLUTION: An aerial display device includes: a display element 20 for displaying an image; an optical element 40 which reflects light from the display element 20 to an opposite direction to the display element 20 for image-forming an aerial image in the air; a sensing element 50 which forms a detection region in a space region crossing the aerial image, and detects an object in the detection region; and a correction unit 70D which corrects detection timing at which the object is detected by the sensing element 50. The detection region includes a first partial region 80 being from an end at the sensing element 50 side of the aerial image to a crossing line of the aerial image and the detection region, and a second partial region 81 being from the crossing line to a far side end of the aerial image to the sensing element 50. When the detection position of the sensing element 50 is the second partial region 81, the correction unit 70D retards the detection timing.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This invention relates to an aerial display device. [Background technology]

[0002] Aerial display devices capable of displaying images and videos as aerial images are being researched and are expected to be a new human-machine interface. For example, an aerial display device may include a two-sided corner reflector array in which two-sided corner reflectors are arranged in an array, reflecting light emitted from the display surface of a display element and forming a real image in the air. The display method using a two-sided corner reflector array is aberration-free and can display a real image (aerial image) in a plane-symmetrical position.

[0003] Patent Document 1 discloses an optical element in which a transparent rectangular prism protruding from the surface of a transparent flat plate is used as a two-sided corner reflector, and multiple rectangular prisms are arranged in an array on a plane. Patent Document 2 discloses an optical element in which each of the first and second light control panels is formed by arranging multiple planar light reflecting parts perpendicularly inside a transparent flat plate, and the first and second light control panels are arranged so that their planar light reflecting parts are orthogonal to each other. The optical elements of Patent Documents 1 and 2 generate an aerial image by reflecting light emitted from a display element twice on orthogonal reflective surfaces.

[0004] Observers can touch the aerial image displayed by the aerial display device without touching the device itself. The sensing elements in the aerial display device detect objects present in the area of ​​the aerial image and recognize when the observer touches the image. The sensing elements are required to detect the observer's touch operation on the aerial image with greater accuracy. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2011-191404 [Patent Document 2] Japanese Patent Publication No. 2011-175297 [Overview of the project] [Problems that the invention aims to solve]

[0006] This invention provides an aerial display device that can more accurately detect touch operations performed by an observer on an aerial image. [Means for solving the problem]

[0007] According to a first aspect of the present invention, an aerial display device is provided, comprising: a display element for displaying an image; an optical element arranged to receive light from the display element and reflecting the light from the display element to the opposite side of the display element to form an aerial image in the air; a sensing element arranged on one side of the optical element in a first direction and forming a detection region in a spatial region intersecting the aerial image, and detecting an object within the detection region; and a correction unit for correcting the detection timing when the object is detected by the sensing element, wherein the detection region includes a first partial region from the end of the aerial image on the sensing element side to the intersection line of the aerial image and the detection region, and a second partial region from the intersection line to the end of the aerial image on the side farther from the sensing element, and the correction unit delays the detection timing when the detection position by the sensing element is in the second partial region.

[0008] According to a second aspect of the present invention, the correction unit delays the detection timing by a specific time and sets the angle between the optical axis of the sensing element and the aerial image to θ sIf Ds is the vertical distance from the sensing element to the top surface of the optical element, Di is the vertical distance from the top surface of the optical element to the aerial image, Dp is the vertical distance from the aerial image to the object, Ld is the length of the aerial image in the first direction, Ls is the horizontal distance from the end of the aerial image on the sensing element side to the sensing element, Li is the horizontal distance from the end of the aerial image on the sensing element side to the intersection line, Lp is the horizontal distance from the intersection line to the object, v is the velocity of the object, and tp is the specific time, then the specific time tp satisfies the following equation: tp = ((Lp + Li + Ls)tanθ s -(Di+Ds)) / v An aerial display device according to the first embodiment is provided.

[0009] According to a third aspect of the present invention, an aerial display device according to the first aspect is provided, wherein the correction unit delays the detection timing by a certain period of time.

[0010] According to a fourth aspect of the present invention, an aerial display device according to the first aspect is provided, wherein the sensing element includes a light-emitting unit that emits light toward the detection area and a light-receiving unit that receives reflected light reflected by the object.

[0011] According to a fifth aspect of the present invention, an aerial display device according to the first aspect is provided, wherein the optical element includes a planar substrate and a plurality of optical elements provided below the substrate, each extending in a second direction perpendicular to the first direction and arranged in the first direction, and each of the plurality of optical elements is inclined with respect to the normal direction of the substrate and has an incident surface and a reflective surface that are in contact with each other.

[0012] According to a sixth aspect of the present invention, an aerial display device according to the first aspect is provided, further comprising an orientation control element disposed between the display element and the optical element, which transmits the oblique light component of the light from the display element.

[0013] According to the seventh aspect of the present invention, there is provided an aerial display device according to the sixth aspect, wherein the alignment control element includes a plurality of transparent members and a plurality of light-shielding members arranged alternately, and the plurality of light-shielding members are inclined with respect to the normal line of the alignment control element.

[0014] According to the eighth aspect of the present invention, there is provided an aerial display device according to the first aspect, wherein the display element and the optical element are arranged parallel to each other.

Advantages of the Invention

[0015] According to the present invention, it is possible to provide an aerial display device capable of more accurately detecting a touch operation on an aerial image by an observer.

Brief Description of the Drawings

[0016] [Figure 1] FIG. 1 is a perspective view of an aerial display device according to the first embodiment of the present invention. [Figure 2] FIG. 2 is a side view of the aerial display device shown in FIG. 1 in the XZ plane. [Figure 3] FIG. 3 is a perspective view for explaining the appearance of the aerial display device. [Figure 4A] FIG. 4A is a plan view of the alignment control element shown in FIG. 1. [Figure 4B] FIG. 4B is a cross-sectional view of the alignment control element taken along the line A-A' in FIG. 4A. [Figure 5] FIG. 5 is a perspective view of the optical element shown in FIG. 1. [Figure 6] FIG. 6 is a diagram for explaining the configuration of the sensing element. [Figure 7] FIG. 7 is a block diagram of the aerial display device. [Figure 8] FIG. 8 is a perspective view for explaining the state of light reflection in the optical element. [Figure 9] FIG. 9 is a side view in the XZ plane for explaining the state of light reflection in the optical element. [Figure 10] FIG. 10 is a side view in the YZ plane for explaining the state of light reflection in the optical element. [Figure 11] Figure 11 illustrates the angular conditions of the incident and reflective surfaces in an optical element. [Figure 12] Figure 12 illustrates the detection area formed by the sensing element. [Figure 13] Figure 13 illustrates how an observer touches an aerial image. [Figure 14] Figure 14 is a schematic diagram illustrating the correction operation of the detection timing. [Figure 15] Figure 15 is a flowchart illustrating the overall operation of the aerial display device. [Figure 16] Figure 16 is a flowchart illustrating the overall operation of the aerial display device according to the second embodiment of the present invention. [Modes for carrying out the invention]

[0017] The embodiments will be described below with reference to the drawings. However, the drawings are schematic or conceptual, and the dimensions and proportions in each drawing are not necessarily the same as those in reality. Furthermore, even when the same part is represented between drawings, the relationship between dimensions and proportions may be represented differently. In particular, the embodiments shown below are illustrative examples of devices and methods for realizing the technical concept of the present invention, and the technical concept of the present invention is not determined by the shape, structure, arrangement, etc. of the components. In the following description, elements having the same function and configuration are denoted by the same reference numerals, and redundant descriptions are omitted.

[0018] [1] First Embodiment [1-1] Configuration of the aerial display device 1 Figure 1 is a perspective view of an aerial display device 1 according to the first embodiment of the present invention. In Figure 1, the X direction is the direction along one side of the aerial display device 1, the Y direction is the direction perpendicular to the X direction in the horizontal plane, and the Z direction is the direction perpendicular to the XY plane (also called the normal direction). Figure 2 is a side view of the aerial display device 1 shown in Figure 1 in the XZ plane. Figure 3 is a perspective view illustrating the external appearance of the aerial display device 1.

[0019] The aerial display device 1 is a device that displays images (including videos). The aerial display device 1 displays an aerial image in the air above its own light-emitting surface. The light-emitting surface of the aerial display device 1 refers to the upper surface of the uppermost component among the multiple components that constitute the aerial display device 1 and are arranged in the optical path. The aerial image is a real image formed in the air.

[0020] The aerial display device 1 comprises an illumination element (also called a backlight) 10, a display element 20, an orientation control element 30, an optical element 40, a sensing element 50, and a housing 60. The illumination element 10, the display element 20, the orientation control element 30, and the optical element 40 are arranged in this order along the Z direction and are arranged parallel to each other. The illumination element 10, the display element 20, the orientation control element 30, and the optical element 40 are fixed in specific positions by fixing members (not shown) with a specific distance between them. In Figure 3, the illumination element 10, the display element 20, the orientation control element 30, and the optical element 40 are collectively referred to as the display module 2. In Figure 3, for simplification, only the display element 20 and the optical element 40 of the display module 2 are shown.

[0021] The illumination element 10 emits illumination light and directs this illumination light toward the display element 20. The illumination element 10 comprises a light source unit 11, a light guide plate 12, and a reflective sheet 13. The illumination element 10 is, for example, a side-light type illumination element. The illumination element 10 constitutes a surface light source. The illumination element 10 may be configured so that the light intensity peaks in an oblique direction at an angle θ1, as described later.

[0022] The light source unit 11 is positioned to face the side of the light guide plate 12. The light source unit 11 emits light toward the side of the light guide plate 12. The light source unit 11 includes a plurality of light-emitting elements, such as white LEDs (Light Emitting Diodes). The light guide plate 12 guides the illumination light from the light source unit 11 and emits the illumination light from its upper surface. The reflective sheet 13 reflects the illumination light emitted from the bottom surface of the light guide plate 12 back toward the light guide plate 12. The illumination element 10 may have members (including a prism sheet and a diffusion sheet) on the upper surface of the light guide plate 12 to improve optical properties.

[0023] The display element 20 is a transmissive display element. The display element 20 is composed of, for example, a liquid crystal display element. The driving mode of the display element 20 is not particularly limited, and TN (Twisted Nematic) mode, VA (Vertical Alignment) mode, or homogeneous mode can be used. The display element 20 receives illumination light emitted from the illumination element 10. The display element 20 transmits the illumination light from the illumination element 10 and performs light modulation. Then, the display element 20 displays a specific image on its screen.

[0024] The orientation control element 30 has the function of reducing unwanted light. Unwanted light is light components that do not contribute to the generation of an aerial image and includes light components that are transmitted through the optical element 40 in the normal direction. The orientation control element 30 is configured to transmit light components within a predetermined angular range centered on an oblique direction at an angle θ1 with respect to the normal direction, while blocking light components outside of the above angular range. The area of ​​the orientation control element 30 is set to be greater than or equal to the area of ​​the display element 20. The detailed configuration of the orientation control element 30 will be described later.

[0025] The optical element 40 reflects light incident from the bottom side to the top side. It also reflects incident light obliquely incident from the bottom side, for example, in the front direction (normal direction). The area of ​​the optical element 40 is set to be greater than or equal to the area of ​​the display element 20. The detailed configuration of the optical element 40 will be described later. The optical element 40 forms an aerial image 3 in the air. The aerial image 3 is parallel to the element plane of the optical element 40 and is a two-dimensional image. The element plane refers to a virtual plane that extends in the in-plane direction of the optical element 40. The element plane has the same meaning as "in-plane". The same meaning applies to the element planes of other elements. An observer 4 standing in front of the optical element 40 can see the aerial image 3.

[0026] The sensing element 50 is positioned at approximately the same level as the optical element 40 and on one side of the optical element 40 in the X direction. In other words, the sensing element 50 is positioned on one side of the optical element 40 in the X direction. The sensing element 50 is positioned below the aerial image 3 such that the light emitted from the sensing element 50 crosses the aerial image 3 diagonally. The sensing element 50 is fixed in a specific position by a fixing member (not shown).

[0027] The sensing element 50 forms a detection area in a two-dimensional spatial region that includes part or all of the aerial image 3 generated by the aerial display device 1. The sensing element 50 detects objects present in the detection area. The sensing element 50 emits infrared light into the detection area and detects the reflected light reflected by the object. The sensing element 50 includes a light-emitting unit that emits infrared light toward the detection area and a light-receiving unit (sensor) that detects the reflected light reflected by the object. For example, the sensing element 50 is composed of a line sensor in which multiple light-emitting elements and multiple light-receiving elements are arranged alternately in a line. The line sensor is capable of scanning space in a line using infrared light and can scan a two-dimensional space consisting of the direction in which the multiple light-emitting elements are arranged and the direction in which the light travels. The direction of the infrared light emitted by the sensing element 50 can be set as appropriate.

[0028] The housing 60 houses the illumination element 10, the display element 20, the orientation control element 30, the optical element 40, and the sensing element 50. The housing 60 has an opening 61 at the top that exposes the optical element 40. The sensing element 50 is mounted on one side of the housing 60. The housing 60 has an opening 62 at the top through which light emitted from the sensing element 50 passes. The sensing element 50 may not be housed in the housing 60 but may be placed at a specific location outside the housing 60. The sensing element 50 can be placed at a location that is optimal for detection operation.

[0029] [1-1-1] Configuration of the orientation control element 30 Figure 4A is a plan view of the orientation control element 30 shown in Figure 1. Figure 4B is a cross-sectional view of the orientation control element 30 along line AA' in Figure 4A.

[0030] The substrate 31 is planar in the XY plane and has a rectangular parallelepiped. The substrate 31 transmits light.

[0031] Multiple transparent members 33 are provided on the base material 31, each extending in the Y direction and aligned in the X direction. Additionally, multiple light-shielding members 34 are provided on the base material 31, each extending in the Y direction and aligned in the X direction. The multiple transparent members 33 and the multiple light-shielding members 34 are arranged alternately so that adjacent members are in contact with each other.

[0032] Multiple transparent members 33 and multiple light-shielding members 34 are provided on a base material 32. The base material 32 is planar in the XY plane and has a rectangular parallelepiped shape. The base material 32 transmits light.

[0033] The transparent member 33 extends in an oblique direction at an angle θ1 with respect to the normal direction of the base material 31 in the XZ plane. The transparent member 33 is a parallelogram with its side surface inclined at an angle θ1 in the XZ plane. The transparent member 33 transmits light.

[0034] The light-shielding member 34 extends in an oblique direction at an angle θ1 with respect to the normal direction of the base material 31 in the XZ plane. The light-shielding member 34 is a parallelogram with its side surface inclined by an angle θ1 in the XZ plane. The light-shielding member 34 blocks light. The thickness of the light-shielding member 34 is set to be thinner than the thickness of the transparent member 33.

[0035] The two adjacent light-shielding members 34 are arranged so that their ends slightly overlap in the Z direction.

[0036] The base materials 31, 32, and transparent member 33 are made of glass or a transparent resin (including acrylic resin). The light-shielding member 34 is made of, for example, a resin mixed with a black dye or pigment.

[0037] The orientation control element 30 may be constructed by omitting one or both of the base materials 31 and 32. The function of the orientation control element 30 can be realized if multiple transparent members 33 and multiple light-shielding members 34 are arranged alternately.

[0038] The orientation control element 30 configured in this way can transmit display light such that the light intensity in the oblique direction at an angle θ1 with respect to the normal direction is at its peak. For example, the orientation control element 30 is configured to block light components outside the range of 30°±30° with respect to the normal direction. Preferably, the orientation control element 30 is configured to block light components outside the range of 30°±20° with respect to the normal direction.

[0039] As a modified example, the orientation control element 30 may be placed between the illumination element 10 and the display element 20. Alternatively, the aerial display device 1 may be configured without the orientation control element 30.

[0040] [1-1-2] Configuration of the optical element 40 Figure 5 is a perspective view of the optical element 40 shown in Figure 1. Figure 5 also shows an enlarged view of a part of the optical element 40. The enlarged view in Figure 5 is a side view in the XZ plane.

[0041] The optical element 40 comprises a substrate 41 and a plurality of optical elements 42. The substrate 41 is planar in the XY plane and has a rectangular parallelepiped.

[0042] Multiple optical elements 42 are provided on the bottom surface of the base material 41. Each of the multiple optical elements 42 is composed of a triangular prism. The optical elements 42 are arranged such that three sides of the triangular prism are parallel to the XY plane, and one side is in contact with the base material 41. Each of the multiple optical elements 42 extends in the Y direction and is arranged in a line in the X direction. In other words, the multiple optical elements 42 have a sawtooth shape in the XZ plane.

[0043] Each of the multiple optical elements 42 has an incident surface 43 and a reflecting surface 44. When viewed from the Y direction, the left side is the incident surface 43 and the right side is the reflecting surface 44. The incident surface 43 is the surface to which light from the display element 20 is incident. The reflecting surface 44 is the surface that reflects light incident on the incident surface 43 from the outside within the optical element 42. The angle between the incident surface 43 and the reflecting surface 44 is θ p It has.

[0044] The base material 41 and the optical element 42 are made of a transparent material. The optical element 42 is formed integrally with the base material 41, for example, using the same transparent material as the base material 41. Alternatively, the base material 41 and the optical element 42 may be formed separately, and the optical element 42 may be bonded to the base material 41 using a transparent adhesive. As the transparent material constituting the base material 41 and the optical element 42, glass or a transparent resin (including acrylic resin) can be used.

[0045] The optical element 40 configured in this way reflects incident light internally to form a real image in the air. The optical element 40 also forms an aerial image 3 at a position directly in front of the element surface.

[0046] [1-1-3] Configuration of the sensing element 50 Figure 6 is a diagram illustrating the configuration of the sensing element 50. Figure 6(a) is a plan view, and Figure 6(b) is a side view.

[0047] The sensing element 50 comprises a substrate 53, a plurality of light-emitting elements 54, a plurality of light-receiving elements 55, a plurality of lenses 56, and a case 57. The number of light-emitting elements 54, light-receiving elements 55, and lenses 56 shown in Figure 6 is just an example. The plurality of light-emitting elements 54 are collectively referred to as the light-emitting section 51, and the plurality of light-receiving elements 55 are collectively referred to as the light-receiving section 52.

[0048] Multiple light-emitting elements 54 and multiple light-receiving elements 55 are mounted on a substrate 53. The substrate 53 includes multiple wiring layers (not shown). The multiple light-emitting elements 54 and multiple light-receiving elements 55 are arranged alternately.

[0049] Each of the multiple light-emitting elements 54 emits laser light. The laser light is, for example, infrared light that does not stimulate the user's vision and is unaffected by visible light. The light-emitting elements 54 are composed of, for example, laser diodes.

[0050] Each of the multiple light-receiving elements 55 detects the laser light (infrared light) reflected from the object. The light-receiving elements 55 are composed of, for example, photodiodes.

[0051] Multiple lenses 56 are provided in accordance with multiple light-emitting elements 54 and are positioned above the multiple light-emitting elements 54. The multiple lenses 56 are arranged in a line in one direction. Each of the multiple lenses 56 is, for example, a plano-convex lens. The multiple lenses 56 have the function of focusing the laser light emitted from the multiple light-emitting elements 54 in a forward direction. In addition, the multiple lenses 56 have the function of focusing the laser light reflected from the object.

[0052] The case 57 houses the substrate 53, a plurality of light-emitting elements 54, a plurality of light-receiving elements 55, and a plurality of lenses 56. The case 57 has an opening at the top that exposes the lenses 56. In Figure 6, only the external shape of the case 57 is shown in a simplified form.

[0053] [1-1-4] Block configuration of the aerial display device 1 Figure 7 is a block diagram of the aerial display device 1. The aerial display device 1 comprises a control unit 70, a storage unit 71, an input / output interface (input / output IF) 72, a display unit 73, a sensing element 50, and an input unit 74. The control unit 70, the storage unit 71, and the input / output interface 72 are connected to each other via a bus 75.

[0054] The input / output interface 72 is connected to the display unit 73, the sensing element 50, and the input unit 74. The input / output interface 72 performs interface processing for each of the display unit 73, the sensing element 50, and the input unit 74 according to a predetermined standard.

[0055] The display unit 73 includes an illumination element 10 and a display element 20. The display unit 73 displays an image.

[0056] The sensing element 50 comprises a light-emitting unit 51 and a light-receiving unit 52. The light-emitting unit 51 emits infrared light toward the detection area. The light-receiving unit 52 detects the reflected light reflected by the object.

[0057] The control unit 70 is composed of one or more processors, such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 70 implements various functions by executing programs stored in the memory unit 71. The control unit 70 includes a display processing unit 70A, an information processing unit 70B, a detection position calculation unit 70C, and a detection timing correction unit 70D.

[0058] The display processing unit 70A controls the operation of the display unit 73 (specifically, the illumination element 10 and the display element 20). The display processing unit 70A controls the on and off of the illumination element 10. The display processing unit 70A transmits an image signal to the display element 20 and causes the display element 20 to display an image.

[0059] The information processing unit 70B generates the image to be displayed by the aerial display device 1. The information processing unit 70B can use image data stored in the storage unit 71. The information processing unit 70B may also acquire image data from an external source using a communication function (not shown).

[0060] The detection position calculation unit 70C controls the operation of the sensing element 50. The detection position calculation unit 70C controls the light-emitting unit 51 included in the sensing element 50 to emit infrared light, thereby forming a detection area consisting of infrared light in a predetermined spatial region. The detection position calculation unit 70C calculates the position of the object based on multiple detection signals sent from the light-receiving unit 52 included in the sensing element 50.

[0061] The detection timing correction unit 70D corrects the detection timing detected by the sensing element 50 and the detection position calculation unit 70C when the sensing element 50 and the detection position calculation unit 70C detect an object. The detection timing correction unit 70D delays the detection timing detected by the sensing element 50 and the detection position calculation unit 70C by a specific time if the detection position detected by the sensing element 50 and the detection position calculation unit 70C is a specific detection sub-region within the detection area.

[0062] The storage unit 71 includes non-volatile storage devices such as ROM (Read Only Memory), HDD (Hard Disk Drive), and SSD (Solid State Drive), and volatile storage devices such as RAM (Random Access Memory) and registers. The storage unit 71 stores the program executed by the control unit 70. The storage unit 71 stores various data necessary for controlling the control unit 70. The storage unit 71 stores the image data displayed by the aerial display device 1.

[0063] The input unit 74 includes, for example, a touch panel or buttons, and receives information entered by the user. The information processing unit 70B can select an image to display on the display unit 73 based on the information received by the input unit 74.

[0064] [1-2] Basic operation of the aerial display device 1 Next, the basic operation of the aerial display device 1 configured as described above will be explained.

[0065] The arrows in Figure 2 indicate the optical path. As shown in Figure 2, light emitted from any point "o" on the display element 20 enters the orientation control element 30. Of the light emitted from the display element 20, the optical component at angle θ1 (including the optical component within a predetermined angular range centered on angle θ1) passes through the orientation control element 30. The light that has passed through the orientation control element 30 enters the optical element 40. The optical element 40 reflects the incident light to the opposite side of the orientation control element 30, forming an aerial image 3 in the air.

[0066] Figure 8 is a perspective view illustrating the reflection of light in the optical element 40. Figure 9 is a side view of the XZ plane illustrating the reflection of light in the optical element 40. Figure 9 shows the optical element 40 as seen by observer 4 with both eyes (i.e., the line connecting both eyes) parallel to the X direction. Figure 10 is a side view of the YZ plane illustrating the reflection of light in the optical element 40. Figure 10 shows the optical element 40 as seen by observer 4 with both eyes parallel to the Y direction.

[0067] Light emitted from any point "o" of the display element 20 enters the incident surface 43 of the optical element 40 and reaches the reflective surface 44. Light that reaches the reflective surface 44 at an angle greater than the critical angle with respect to the normal direction of the reflective surface 44 is totally reflected by the reflective surface 44 and is emitted from the plane opposite to the side of the optical element 42 of the optical element 40. The critical angle is the smallest angle of incidence beyond which total reflection occurs. The critical angle is the angle with respect to the perpendicular to the incident surface.

[0068] In the XZ plane of Figure 9, light emitted from point "o" is totally reflected by the reflective surface 44 of the optical element 42, and the light is imaged in the air to generate an aerial image.

[0069] In the YZ plane of Figure 10, the light emitted from point "o" is not reflected by the reflective surface 44 of the optical element 42, and therefore does not form an image in the air, thus not contributing to the generation of an aerial image.

[0070] That is, the condition for the observer 4 to visually recognize the aerial image is that both eyes of the observer 4 are parallel to the X direction or in a state close thereto (for example, ±10 degrees with respect to the X direction). Further, when the observer 4 moves the viewpoint along the Y direction with both eyes parallel to the X direction or in a state close thereto, the aerial image can always be recognized.

[0071] FIG. 11 is a diagram for explaining the angular conditions of the incident surface 43 and the reflection surface 44 in the optical element 40.

[0072] Let the angle of the incident surface 43 with respect to the Z direction (direction perpendicular to the element surface) be θ2, the angle of the reflection surface 44 with respect to the Z direction be θ3, and the angle formed by the incident surface 43 and the reflection surface 44 be θ p be. The angle θ p is represented by the following formula (1). θ p = θ2 + θ3 ··· (1)

[0073] The light emitted from the orientation control element 30 at an angle θ1 is incident on the incident surface 43. Let the refractive index of the material of the optical element 40 be n p , and the refractive index of air be 1. Let the incident angle on the incident surface 43 be θ4 and the refraction angle be θ5. Let the incident angle on the reflection surface 44 be θ6 and the reflection angle be θ7 (= θ6). Let the incident angle on the upper surface of the optical element 40 be θ8 and the refraction angle be θ9. The refraction angle θ9 is the emission angle. The emission angle θ9 is represented by the following formula (2). θ9 = sin -1 (n p * sin (sin -1 ((1 / n p ) * sin (90° - (θ1 + θ2)) + θ2 + 2θ3 - 90°)) ··· (2)

[0074] The critical angle on the reflection surface 44 is represented by the following formula (3). Critical angle < θ6 (= θ7) Critical angle = sin -1 (1 / n p ) ··· (3)

[0075] In other words, the angle of incidence θ6 at the reflective surface 44 is set to be greater than the critical angle at the reflective surface 44. To put it another way, the angle θ3 of the reflective surface 44 is set such that the angle of incidence of light incident on the reflective surface 44 is greater than the critical angle.

[0076] Furthermore, the light incident on the incident surface 43 is set so that it is not totally reflected at the incident surface 43. In other words, the angle θ2 of the incident surface 43 is set so that the angle of incidence of the light incident on the incident surface 43 is smaller than the critical angle.

[0077] The angle between the element surface of the optical element 40 and the surface of the aerial image 3, and the distance between the element surface of the optical element 40 and the surface of the aerial image 3, can be adjusted by optimally setting the angle θ1 of the light incident on the optical element 40, the refractive index of the optical element 40, the angle θ2 of the incident surface 43 of the optical element 40, and the angle θ3 of the reflective surface 44 of the optical element 40.

[0078] [1-3] Object detection operation Next, we will explain the object detection process.

[0079] Figure 12 illustrates the detection region 58 formed by the sensing element 50. Figure 12 corresponds to a side view of the aerial display device 1 in the XZ plane.

[0080] The sensing element 50 is positioned below the level at which the aerial image 3 is formed. For example, the sensing element 50 is positioned at approximately the same level as the optical element 40, or slightly below the optical element 40. The sensing element 50 is positioned such that its optical axis (i.e., the direction of light emission) points diagonally upward.

[0081] The sensing element 50 emits infrared light toward the aerial image 3, forming a detection region 58 consisting of infrared light. The detection region 58 consists of a two-dimensional spatial region. The detection region 58 intersects the aerial image 3 diagonally. For example, the detection region 58 intersects the aerial image 3 approximately at its center in the X direction.

[0082] Figure 13 illustrates how observer 4 touches the aerial image 3. The aerial image 3 is, for example, a push button. Observer 4 touches the aerial image 3 with their finger 4A.

[0083] The shape of the aerial image is predetermined as the image to be displayed by the aerial display device 1, and information regarding the shape of the aerial image is stored in the storage unit 71 in association with the information of the image displayed by the aerial display device 1. Information regarding the area occupied by the aerial image 3 within the detection area 58 when viewed from the Z direction (called the determination area) is stored in the storage unit 71. If the position of the detected object is within the determination area, the control unit 70 determines that the aerial image 3 has been touched. When displaying multiple aerial images at different positions, a determination area is set for each of the multiple aerial images.

[0084] When observer 4's finger 4A is located on the side of the aerial image 3 closer to the sensing element 50, observer 4's finger 4A reaches the detection area 58 after passing through the aerial image 3. On the other hand, when observer 4's finger 4A is located on the side of the aerial image 3 further away from the sensing element 50, observer 4's finger 4A reaches the aerial image 3 after passing through the detection area 58. Thus, when observer 4 touches the aerial image 3 with finger 4A, the timing of touch operation detection differs depending on the position of observer 4's finger 4A.

[0085] In the XZ plane, the region of the detection area 58 from the edge of the aerial image 3 on the sensing element 50 side to the intersection line between the aerial image 3 and the detection area 58 is called the first detection sub-region 80, and the region from the intersection line to the edge of the aerial image 3 farther from the sensing element 50 is called the second detection sub-region 81. When observer 4 touches the aerial image 3 corresponding to the first detection sub-region 80, the sensing element 50 detects the observer's touch operation after the observer's finger 4A has passed over the aerial image 3. When observer 4 touches the aerial image 3 corresponding to the second detection sub-region 81, the sensing element 50 detects the observer's touch operation before the observer's finger 4A reaches the aerial image 3. In other words, with respect to the aerial image 3, the detection timing in the second detection sub-region 81 is earlier than the detection timing in the first detection sub-region 80.

[0086] In this embodiment, if the position of observer 4's finger 4A is within the second detection area 81, the timing of detecting the touch operation on the aerial image 3 by observer 4 is corrected.

[0087] Figure 14 is a schematic diagram illustrating the correction operation of the detection timing. The angle between the XY plane and the detection area 58 is θ. s Let's assume that the XY plane is a plane parallel to the aerial image 3. The angle between the optical axis of the sensing element 50 and the XY plane is θ. s The sensing element 50 has its light-emitting surface at an angle θ. s It is positioned so that it is angled upwards. Angle θ s The angle is greater than 0 and less than or equal to 60 degrees. The light-emitting surface of the sensing element 50 is the surface from which light is emitted when the sensing element 50 is viewed as a rectangular parallelepiped.

[0088] Ds is the vertical distance from the light-emitting surface of the sensing element 50 to the top surface of the optical element 40, Di is the vertical distance from the top surface of the optical element 40 to the aerial image 3, and Dp is the vertical distance from the aerial image 3 to the pressed position. The top surface of the optical element 40 has the same meaning as the light-emitting surface of the aerial display device 1. The pressed position is the position where the observer 4's finger 4A reaches the detection area 58 when the observer 4 touches the aerial image 3 with their finger 4A.

[0089] Let Ld be the length of the aerial image 3 in the X direction, Ls be the horizontal distance from the end of the aerial image 3 on the sensing element 50 side to the light emitting surface of the sensing element 50, Li be the horizontal distance from the end of the aerial image 3 on the sensing element 50 side to the intersection line of the aerial image 3 and the detection area 58, and Lp be the horizontal distance from the intersection line to the pressed position. Here, the length (size) of the aerial image 3 refers to the maximum area in which the aerial image 3 can be formed. The distance Dp is expressed by the following equation (4). Dp = (Lp + Li + Ls) tanθ s -(Di+Ds) ···(4)

[0090] Let v be the speed of finger 4A when pressed, t be the time, and D be the distance. The distance D is expressed by the following equation (5). D = vt ... (5) Speed ​​v is the average speed of a person's finger when pressing a button, and is preset. The information about speed v is stored in the memory unit 71.

[0091] If the time taken to travel a distance Dp is denoted as tp, then from equations (4) and (5), the time tp can be expressed by the following equation (6). tp = ((Lp + Li + Ls)tanθ s -(Di+Ds)) / v ···(6)

[0092] The detection timing correction unit 70D delays the detection timing by the time tp calculated by equation (6) when an object is present in the second detection area 81 described above. This makes it possible to make the timing when the observer 4 touches the aerial image 3 and the timing of the touch operation detection approximately the same.

[0093] [1-4] Overall flow of operations Next, we will explain the overall operation flow of the aerial display device 1. Figure 15 is a flowchart illustrating the overall operation of the aerial display device 1.

[0094] The control unit 70 displays the aerial image 3 (step S100). The display processing unit 70A displays the image on the screen of the display element 20. The optical element 40 reflects the light from the display element 20 and forms an image of the aerial image 3 in the air.

[0095] Next, the sensing element 50 performs a sensing operation (step S101). The light-emitting part 51 included in the sensing element 50 emits infrared light into the detection area 58 that intersects with the area where the aerial image 3 is displayed.

[0096] Next, the sensing element 50 monitors whether or not an object exists within the detection area 58 (step S102). That is, the light receiving unit 52 included in the sensing element 50 monitors the infrared light reflected by the object.

[0097] If the sensing element 50 detects an object (S102=Yes), the detection position calculation unit 70C calculates the detection position within the detection area 58 based on the signal detected by the sensing element 50 (step S103). The detection position calculation unit 70C calculates the position of the object based on the time the light-emitting unit 51 emitted light and the time the light-receiving unit 52 received the reflected light.

[0098] Next, the detection timing correction unit 70D determines whether the detection position in step S103 is in the first detection portion area 80 or the second detection portion area 81 of the detection area 58 (step S104). The information of the first detection portion area 80 and the second detection portion area 81 is stored in the storage unit 71 in association with the information of the aerial image 3.

[0099] If the detection position is the second detection area 81 (step S104 = second detection area), the detection timing correction unit 70D delays the detection timing at which the sensing element 50 detects the object by a specific time (step S105). The specific time in step S105 is the time tp calculated by the above formula (6).

[0100] Next, the detection timing correction unit 70D determines that the aerial image 3 has been touched at the correction timing delayed in step S105 (step S106). After that, the aerial display device 1 performs an operation corresponding to the touch operation of the observer 4.

[0101] If the detection position is in the first detection area 80 (step S104 = first detection area), the detection timing correction unit 70D determines that the aerial image 3 has been touched at the detection timing when the sensing element 50 detects the object (step S106). In other words, the detection timing correction unit 70D does not delay the detection timing. Subsequently, the aerial display device 1 performs an operation corresponding to the touch operation of the observer 4.

[0102] [1-5] Effects of the first embodiment In the first embodiment, the sensing element 50 is positioned on one side of the optical element 40 and forms a detection region 58 consisting of infrared light that diagonally crosses the aerial image 3. The detection timing correction unit 70D delays the detection timing by a specific time tp if the detection position of the object detected by the sensing element 50 is in a region (second detection region 81) that is further from the sensing element 50 than the intersection line where the aerial image 3 and the detection region 58 intersect.

[0103] Therefore, according to the first embodiment, the timing of when observer 4 touches the aerial image 3 can be detected more accurately. Consequently, an aerial display device 1 that can more accurately detect the touch operation of observer 4 on the aerial image 3 can be realized.

[0104] Furthermore, the aerial display device 1 can display an aerial image 3 in the air by reflecting light emitted from the display element 20 with the optical element 40. In addition, the aerial display device 1 can display the aerial image 3 parallel to the element surface of the optical element 40 in its frontal direction. Moreover, it is possible to realize an aerial display device 1 that can improve display quality.

[0105] Furthermore, when observer 4 views the optical element 40 with both eyes parallel to or nearly parallel to the X direction (i.e., the direction in which the multiple optical elements 42 are aligned), observer 4 can perceive the aerial image. Also, when observer 4 moves their viewpoint along the Y direction with both eyes parallel to or nearly parallel to the X direction, they can always perceive the aerial image. In addition, a wider field of view can be achieved when observer 4's eyes are parallel to or nearly parallel to the X direction.

[0106] Furthermore, multiple elements constituting the aerial display device 1 can be arranged in parallel. This makes it possible to realize an aerial display device 1 that can be miniaturized in the Z direction.

[0107] [2] Second embodiment In the second embodiment, when the detection position of the object is in the second detection sub-region 81 of the detection region 58, the detection timing is delayed by a certain period of time.

[0108] Figure 16 is a flowchart illustrating the overall operation of the aerial display device 1 according to the second embodiment of the present invention. The operation from steps S100 to S104 is the same as in the first embodiment.

[0109] If the detection position is in the second detection area 81 (step S104 = second detection area), the detection timing correction unit 70D delays the detection timing in which the sensing element 50 detects the object by a certain period of time (step S200). The certain period in step S200 is a predetermined time, for example, 1 second. The certain period is set based on the average distance between the aerial image 3 and the second detection area 81. For example, the certain period is calculated by substituting the average distance D between the aerial image 3 and the second detection area 81 into equation (5).

[0110] Next, the detection timing correction unit 70D determines that the aerial image 3 has been touched at the correction timing delayed in step S200 (step S106). After that, the aerial display device 1 performs an operation corresponding to the touch operation of the observer 4.

[0111] According to the second embodiment, special calculations by the control unit 70 can be omitted. This simplifies the processing of the control unit 70. Other effects are the same as in the first embodiment.

[0112] [3] Variant In the above embodiment, the display element 20 and the optical element 40 are arranged in parallel. However, the embodiment is not limited to this, and the display element 20 may be arranged diagonally to the optical element 40. The angle between the display element 20 and the optical element 40 is set to a range greater than 0 degrees and less than 45 degrees. In this modified example, the orientation control element 30 can be omitted.

[0113] The present invention is not limited to the optical element 40 described in the above embodiment, and other types of imaging elements can also be used. For example, the optical element 40 may be configured with a two-sided corner reflector array in which two-sided corner reflectors are arranged in an array. When a two-sided corner reflector array is used, an aerial image is formed at a position symmetrical to the imaging element. In this modified example, the sensing element 50 is arranged so as to form a detection region that intersects the surface on which the aerial image is displayed.

[0114] In the above embodiment, the left side of the optical element 42 is defined as the incident surface 43, and the right side is defined as the reflective surface 44. However, the embodiment is not limited to this, and the incident surface 43 and the reflective surface 44 may be configured in reverse. In this case, the operation of the aerial display device 1 described in the embodiment will also be reversed left and right.

[0115] In the above embodiment, a liquid crystal display element is used as an example for the display element 20, but it is not limited to this. The display element 20 can also be a self-emissive organic EL (electroluminescence) display element or a micro-LED (light-emitting diode) display element. A micro-LED display element is a display element that emits R (red), G (green), and B (blue) light, which constitute the pixels, using LEDs. When a self-emissive display element 20 is used, the illumination element 10 is not required.

[0116] The present invention is not limited to the embodiments described above, and can be modified in various ways during implementation without departing from its essence. Furthermore, each embodiment may be combined as appropriate, and in that case, the combined effects can be obtained. Moreover, the above embodiments include various inventions, and various inventions can be extracted by selecting combinations from the multiple constituent elements disclosed. For example, if the problem can be solved and effects obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiment, then the configuration with these deleted constituent elements can be extracted as an invention. [Explanation of Symbols]

[0117] 1...Aerial display device, 2...Display module, 3...Aerial image, 4...Observer, 10...Illumination element, 11...Light source unit, 12...Light guide plate, 13...Reflective sheet, 20...Display element, 30...Orientation control element, 31...Substrate, 32...Substrate, 33...Transparent member, 34...Light shielding member, 40...Optical element, 41...Substrate, 42...Optical element, 43...Incident surface, 44...Reflective surface, 50...Sensing element, 51...Light emitting unit, 52...Light receiving unit, 53...Substrate, 5 4...Light-emitting element, 55...Light-receiving element, 56...Lens, 57...Case, 58...Detection area, 60...Housing, 61...Aperture, 62...Aperture, 70...Control unit, 70A...Display processing unit, 70B...Information processing unit, 70C...Detection position calculation unit, 70D...Detection timing correction unit, 71...Storage unit, 72...Input / output interface, 73...Display unit, 74...Input unit, 75...Bus, 80...First detection area, 81...Second detection area.

Claims

1. A display element that displays an image, An optical element is positioned to receive light from the display element, reflects the light from the display element to the opposite side of the display element, and forms an aerial image in the air. A sensing element is provided, which is positioned on one side of the optical element in a first direction, forms a detection region in a spatial region that intersects with the aerial image, and detects an object within the detection region. A correction unit that corrects the detection timing at which the object is detected by the sensing element, It is equipped with, The detection region includes a first partial region from the end of the aerial image on the sensing element side to the intersection line between the aerial image and the detection region, and a second partial region from the intersection line to the end of the aerial image on the side farther from the sensing element. The correction unit delays the detection timing when the detection position by the sensing element is in the second partial region. Aerial display device.

2. The correction unit delays the detection timing by a specific time, The angle between the optical axis of the sensing element and the aerial image is θ. ｓ If Ds is the vertical distance from the sensing element to the top surface of the optical element, Di is the vertical distance from the top surface of the optical element to the aerial image, Dp is the vertical distance from the aerial image to the object, Ld is the length of the aerial image in the first direction, Ls is the horizontal distance from the end of the aerial image on the sensing element side to the sensing element, Li is the horizontal distance from the end of the aerial image on the sensing element side to the intersection line, Lp is the horizontal distance from the intersection line to the object, v is the velocity of the object, and tp is the specific time, then the specific time tp satisfies the following equation: _________________________________________________________________ ｓ ________)__________________ The aerial display device according to claim 1.

3. The correction unit delays the detection timing by a certain period of time. The aerial display device according to claim 1.

4. The sensing element includes a light-emitting unit that emits light toward the detection area and a light-receiving unit that receives reflected light reflected by the object. The aerial display device according to claim 1.

5. The optical element includes a planar substrate and a plurality of optical elements provided beneath the substrate, each extending in a second direction perpendicular to the first direction and arranged in the first direction. Each of the plurality of optical elements is inclined with respect to the normal direction of the substrate and has an incident surface and a reflective surface that are in contact with each other. The aerial display device according to claim 1.

6. The system further comprises an orientation control element disposed between the display element and the optical element, which transmits the oblique light component of the light from the display element. The aerial display device according to claim 1.

7. The orientation control element includes a plurality of transparent members and a plurality of light-shielding members arranged alternately. The plurality of light-shielding members are inclined with respect to the normal of the orientation control element. The aerial display device according to claim 6.

8. The display element and the optical element are arranged parallel to each other. The aerial display device according to claim 1.

Images