Aerial image display device

Abstract

To provide an aerial video display apparatus to be used as an aerial touch panel, configured to improve touch operation accuracy.SOLUTION: An aerial video display apparatus includes: a housing 2; a video display unit 3; a beam splitter 4 which reflects a video displayed by the video display unit; a retroreflection sheet 5 which retroreflects the video reflected by the beam splitter; a sensor 7 which detects a position of a tip of a finger of a user in a space area including an aerial video formed by imaging the retroreflected video outside the housing; and a position calculation unit 8 which calculates a position of the pad of a finger closest to the aerial video from the position of the fingertip detected by the sensor. The position calculation unit calculates the position of the pad of the finger closest to the aerial video from the position of the fingertip detected by the sensor, on the basis of an angle formed by a line perpendicular to a display surface of the video display unit and the beam splitter, and a diameter of a hemisphere when assuming that the fingertip is a hemisphere.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This application relates to an aerial video display device.

Background Art

[0002] An aerial video display device is known that does not require special glasses and can display video in the air without using a screen such as fog. For example, an aerial video display device is disclosed that uses a beam splitter and a retroreflective sheet to form an image of the video displayed on the video display unit in a space at a position symmetric to the beam splitter with respect to the beam splitter (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In an aerial image display device, there is a method of using a touch panel as an aerial touch panel that displays a touch panel as an image displayed in the air (hereinafter referred to as an aerial image), and the user operates the touch panel with a finger. The position of the user's finger is detected by a three-dimensional distance sensor installed in the main body of the aerial image display device. The user approaches the finger from above and operates the touch panel displayed as an aerial image. On the other hand, the three-dimensional distance sensor is also used to detect that the user has approached the aerial image display device in order to operate the touch panel. Therefore, the three-dimensional distance sensor is installed at the upper part of the main body of the aerial image display device. Since the three-dimensional distance sensor can only measure the position of the surface of an object visible from this sensor, the three-dimensional distance sensor measures the back of the user's finger, that is, the side with nails. Therefore, the belly of the user's finger, which is the touch position of the touch panel, cannot be directly measured because it is in the shadow of the back of the finger. As a result, there is a problem that a deviation occurs between the position of the back of the finger measured by the three-dimensional distance sensor and the position of the belly of the user's finger, and the accuracy of the touch operation is low.

[0005] This application was made to solve the above problems, and an object of the present invention is to improve the accuracy of touch operations in an aerial image display device used as an aerial touch panel.

Means for Solving the Problems

[0006] The aerial image display device of the present application includes a housing, an image display unit disposed inside the housing, a beam splitter installed at an opening of the housing for reflecting the image displayed on the image display unit, a retroreflective sheet installed inside the housing for retroreflecting the image reflected by the beam splitter, a sensor for detecting the position of the fingertip of the user's finger in the spatial region including the aerial image where the image retroreflected by the retroreflective sheet is imaged outside the housing, and a position calculation unit for calculating the position of the finger belly closest to the aerial image from the position of the fingertip detected by the sensor. The sensor is installed at the upper part of the housing,The position calculation unit calculates the position of the fingertip closest to the aerial image from the position of the fingertip detected by the sensor based on the angle formed between the line perpendicular to the display surface of the video display unit and the beam splitter and the diameter of the hemisphere when the fingertip is assumed to be a hemisphere.

Advantages of the Invention

[0007] In the aerial image display device of the present application, the position calculation unit calculates the position of the fingertip closest to the aerial image from the position of the fingertip detected by the sensor based on the angle formed between the line perpendicular to the display surface of the video display unit and the beam splitter and the diameter of the hemisphere when the fingertip is assumed to be a hemisphere. Therefore, the accuracy of the touch operation can be improved.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0009] Hereinafter, the aerial image display device according to the embodiment for carrying out the present application will be described in detail with reference to the drawings. In each figure, the same reference numerals indicate the same or corresponding parts.

[0010] Embodiment 1. FIG. 1 is a schematic diagram of an aerial video display device according to Embodiment 1. The aerial video display device 1 of this embodiment includes a housing 2, a video display unit 3 disposed inside the housing 2, a beam splitter 4 provided at an opening of the housing 2, and a retroreflective sheet 5 disposed inside the housing 2. The light emitted from the video display unit 3 enters the beam splitter 4, and a part of it is reflected and enters the retroreflective sheet 5. The retroreflective sheet 5 has the property of reflecting the incident light in the incident direction. The retroreflective sheet 5 reflects the light incident from the beam splitter 4 toward the beam splitter 4. A part of the light reflected from the retroreflective sheet 5 toward the beam splitter 4 passes through the beam splitter 4. In this way, the video displayed on the video display unit 3 passes through the beam splitter 4 and forms an image outside the housing 2 to become an aerial video 6. The user 10 can observe this aerial video 6. For example, a touch panel for operating various devices is displayed as the aerial video 6. The user 10 can perform a touch operation on the touch panel by bringing the finger 10a close to the touch panel displayed as the aerial video 6. The position of the finger 10a can be measured by a three-dimensional distance sensor 7 provided on the housing 2 and detecting the position of an object within the spatial region including the aerial video 6. As the video display unit 3, a liquid crystal display, an organic EL display, a display in which LED elements are arranged, etc. can be used. As the beam splitter 4, a half mirror, a reflective polarizing plate, etc. can be used. As the retroreflective sheet 5, a glass bead type sheet in which spherical transparent glass beads are arranged, a micro prism type sheet in which transparent triangular prisms are arranged, etc. can be used. As the three-dimensional distance sensor 7, for example, a ToF sensor (Time of Flight Sensor), a stereo camera, etc. can be used.

[0011] The user 10 can perform a touch operation by bringing the finger 10a close to the touch panel displayed as the aerial video 6. Since this touch operation is non-contact, it is possible to prevent hand contamination due to contact, the spread of harmful substances through the touch panel, etc.

[0012] The position of the finger 10a of the user 10 is measured by the three-dimensional distance sensor 7. Since this three-dimensional distance sensor 7 is also used to detect that the user 10 has approached the aerial video display device 1 to operate the touch panel, it is usually installed at the upper part of the main body of the aerial video display device 1. Since the three-dimensional distance sensor 7 can only measure the position of the surface of an object visible from this sensor, the ventral side of the finger 10a of the user 10, which is the touch position of the touch panel, is in the shadow of the back of the finger and cannot be directly measured. Therefore, there has been a problem that a deviation occurs between the position of the ventral side of the finger 10a of the user 10 and the position of the back of the finger measured by the three-dimensional distance sensor 7, resulting in low accuracy of the touch operation.

[0013] The aerial video display device 1 of the present embodiment improves the accuracy of the touch operation by calculating the position of the ventral side of the finger 10a, which is the touch position of the touch panel, from the position of the fingertip of the finger 10a measured by the three-dimensional distance sensor 7. Therefore, the aerial video display device 1 of the present embodiment includes a position calculation unit 8 that calculates the position of the ventral side of the finger 10a, which is the touch position of the touch panel, from the position of the fingertip of the finger 10a measured by the three-dimensional distance sensor 7.

[0014] Figure 2 is an explanatory diagram of the coordinate system in the aerial video display device 1 of the present embodiment. As shown in Figure 2, in the present embodiment, a sensor coordinate system and an aerial video coordinate system are defined. The sensor coordinate system is based on the structure of the aerial video display device 1, and the aerial video coordinate system is based on the structure of the virtual image of the aerial video displayed by the aerial video display device 1. The Zs-axis of the sensor coordinate system is in a direction parallel to the beam splitter 4. The Ys-axis of the sensor coordinate system is in a direction perpendicular to the beam splitter 4. The Xs-axis of the sensor coordinate system is in a direction perpendicular to the Zs-axis and the Ys-axis. Note that this sensor coordinate system is obtained by correcting the unique coordinate system of the three-dimensional distance sensor 7 at the mounting position of the three-dimensional distance sensor 7 and converting it into a coordinate system based on the structure of the aerial video display device 1. On the other hand, the Zi-axis of the aerial video coordinate system is in a direction perpendicular to the aerial video 6. The Yi-axis of the aerial video coordinate system is in a direction parallel to the aerial video 6. The Xi-axis of the aerial video coordinate system is in a direction perpendicular to the Zi-axis and the Yi-axis. Here, the Xs-axis of the sensor coordinate system and the Xi-axis of the aerial video coordinate system are parallel to each other and are in a direction perpendicular to the plane of the paper in Figure 1. Therefore, as shown in Figure 2, the aerial video coordinate system is rotated by an angle θ about the Xi-axis with respect to the sensor coordinate system. Since the aerial video 6 is the image displayed on the video display unit 3 displayed at a position symmetric with respect to the beam splitter 4, the angle θ corresponds to the angle formed by the line perpendicular to the display surface of the video display unit 3 and the beam splitter 4. Note that in the present embodiment, a coordinate system obtained by correcting the unique coordinate system of the three-dimensional distance sensor 7 at the mounting position of the three-dimensional distance sensor 7 and converting it into a coordinate system based on the structure of the aerial video display device 1 is used as the sensor coordinate system, but the unique coordinate system of the three-dimensional distance sensor 7 may be used as the sensor coordinate system.

[0015] Next, the measurement position of the finger 10a in the aerial video display device 1 of the present embodiment will be described. FIG. 3 is an explanatory diagram of the position of the finger 10a measured by the three-dimensional distance sensor 7. As shown in FIG. 3(a), when the user 10 operates the touch panel with the finger 10a, the position where the finger 10a touches the touch panel is the part of the fingertip 11 of the finger 10a. The three-dimensional distance sensor 7 is installed at the upper part of the main body of the aerial video display device 1. Therefore, as shown by the thick line in FIG. 3(b), the three-dimensional distance sensor 7 measures the surface on the back side of the user's finger 10a. Here, the three-dimensional distance sensor 7 measures the position of the fingertip 12 at the forefront of the back of the user's finger 10a.

[0016] FIG. 4 is an explanatory diagram of the modeled finger in the aerial video display device 1 of the present embodiment. As shown in FIG. 4, in the present embodiment, it is assumed that the shape of the finger is a combination of a cylinder and a hemisphere. That is, it is assumed that the body part of the finger 10a is a cylinder with a diameter D, and the fingertip part is a hemisphere with a diameter D. And the position where it touches the touch panel when operating the touch panel is a part of the hemisphere at the fingertip part.

[0017] FIG. 5 is an explanatory diagram showing the relationship between the measurement position of the three-dimensional distance sensor 7 and the position where it touches the touch panel in the modeled finger in the present embodiment. FIG. 5 is shown in a coordinate system obtained by rotating the coordinate system shown in FIG. 2 by an angle θ. Since the Xi-axis direction of the aerial video coordinate system is parallel to the Xs-axis direction of the sensor coordinate system, the following explanation will be made in the YZ plane. The three-dimensional distance sensor 7 measures the point located at the outermost end in the Ys-axis direction in the sensor coordinate system of the fingertip of the modeled finger 10a. And let that point be the point (yd, zd) in the aerial video coordinate system. At this time, the tangent direction of the point (yd, zd) is parallel to the Zs-axis direction in the sensor coordinate system. Therefore, as shown in FIG. 5, the angle formed by the line segment connecting the point (yd, zd) and the center C of the hemisphere of the modeled fingertip and the straight line parallel to the Yi-axis is θ.

[0018] When operating the touch panel, the position on the touch panel closest to the finger 10a is a part of the hemispherical body of the modeled fingertip, and as shown in FIG. 5, it is a point located at the outermost end in the Zi-axis direction in the aerial image coordinate system. Let this point be the point (ya, za) in the aerial coordinate system. In the finger modeled in this way, the point (yd, zd) corresponds to the position of the fingertip 12 shown in FIG. 3, and the point (ya, za) corresponds to the position of the finger belly 11 closest to the aerial image shown in FIG. 3. At this time, the following two equations hold between the point (yd, zd) and the point (ya, za).

[0019] ya = yd + D / 2 × cosθ (1) za = zd - D / 2 + D / 2 × sinθ (2)

[0020] In the aerial image display device 1 of the present embodiment, when the angle formed by the line perpendicular to the display surface of the image display unit 3 and the beam splitter 4 is θ, and the shape of the user's fingertip is assumed to be a hemispherical body and the diameter of the hemispherical body is D, the position calculation unit 8 uses the equations (1) and (2) from the position (yd, zd) of the fingertip measured by the three-dimensional distance sensor 7 to calculate the position (ya, za) of the finger belly closest to the touch panel. It is assumed that there is no deviation in the Xs-axis direction between the position of the fingertip 12 and the position of the finger belly 11. The angle formed by the line perpendicular to the display surface of the image display unit 3 and the beam splitter 4 is uniquely determined by the structure of the aerial image display device 1. Also, when the shape of the user's fingertip is assumed to be a hemispherical body, the diameter of the hemispherical body can be determined in advance. In addition, the width of the body part of the user's finger 10a may be measured by the three-dimensional distance sensor 7, and this width may be used as the diameter of the hemispherical body when the shape of the user's fingertip is assumed to be a hemispherical body.

[0021] In the aerial image display device configured in this way, since the position of the finger belly closest to the touch panel can be calculated from the position of the fingertip measured by the three-dimensional distance sensor, the accuracy of the touch operation can be improved.

[0022] FIG. 6 is an explanatory diagram when the angle of the finger changes with respect to the aerial image. As shown in FIG. 6, even when the angle of the finger 10a changes with respect to the aerial image 6, assuming that the shape of the fingertip is a hemisphere, the position of the foremost end of the back of the finger 10a does not change. Therefore, even when the angle of the finger changes with respect to the aerial image, the position of the belly of the finger closest to the touch panel can be calculated using the same formula.

[0023] In the first embodiment, the aerial image display device using a beam splitter and a retroreflective sheet has been described. As other types of aerial image display devices, there are those using an optical plate called a two-sided corner reflector array and those using an optical plate called an orthogonal mirror array. In these types of aerial image display devices, in the aerial image display device shown in FIG. 1, the retroreflective sheet is removed and an optical plate is arranged at the position of the beam splitter. Such an aerial image display device can display an aerial image by forming an image of the image displayed on the image display unit in a space at a position that is plane-symmetric with respect to the optical plate. Also in such a type of aerial image display device, there is a problem that a deviation occurs between the position of the back of the finger measured by the three-dimensional distance sensor and the position of the belly of the user's finger, resulting in low touch operation accuracy. Also in such a type of aerial image display device, the accuracy of the touch operation can be improved by calculating the position of the belly of the finger 10a that becomes the touch position on the touch panel from the position of the fingertip of the finger 10a measured by the three-dimensional distance sensor 7.

[0024] Although exemplary embodiments are described in this application, the various features, aspects, and functions described in the embodiments are not limited to the application of a specific embodiment, but are applicable to the embodiments alone or in various combinations. Therefore, countless variations not illustrated are envisioned within the scope of the technology disclosed in this application specification. For example, it shall include cases where at least one component is deformed, added, or omitted.

Description of Reference Numerals

[0025] 1 Aerial image display device, 2 housing, 3 image display unit, 4 beam splitter, 5 retroreflective sheet, 6 aerial image, 7 three-dimensional distance sensor, 8 position calculation unit, 10 user, 10a finger, 11 finger pad, 12 fingertip.

Claims

1. A housing, a video display unit disposed inside the housing, a beam splitter installed at an opening of the housing and reflecting the video displayed on the video display unit, a retroreflective sheet installed inside the housing and retroreflecting the video reflected by the beam splitter, a sensor that detects the position of the fingertip of a user's finger within a spatial region including a virtual image where the video retroreflected by the retroreflective sheet is imaged outside the housing, and a position calculation unit that calculates the position of the finger pad closest to the virtual image from the position of the fingertip detected by the sensor. The virtual image display device is characterized in that the sensor is installed at the upper part of the housing, the position calculation unit calculates the position of the finger pad closest to the virtual image from the position of the fingertip detected by the sensor based on the angle formed between a line perpendicular to the display surface of the video display unit and the beam splitter and the diameter of the hemisphere when the fingertip is assumed to be a hemisphere.

2. A housing, a video display unit disposed inside the housing, an optical plate installed at an opening of the housing and forming a virtual image of the video displayed on the video display unit in a space outside the housing to display a virtual image, a sensor that detects the position of the fingertip of a user's finger within a spatial region including the virtual image formed by the optical plate, and a position calculation unit that calculates the position of the finger pad closest to the virtual image from the position of the fingertip detected by the sensor. The virtual image display device is characterized in that the optical plate forms a virtual image of the video displayed on the video display unit in a space that is symmetric with respect to the optical plate to display the virtual image, the sensor is installed at the upper part of the housing, the position calculation unit calculates the position of the finger pad closest to the virtual image from the position of the fingertip detected by the sensor based on the angle formed between a line perpendicular to the display surface of the video display unit and the optical plate and the diameter of the hemisphere when the fingertip is assumed to be a hemisphere.

Images