Non-contact input device

Abstract

The present invention provides a non-contact input device that allows input operations to be performed smoothly.　This non-contact input device comprises: a display device that displays a non-contact interface screen; an operation space provided at a position that does not overlap the non-contact interface screen; and a detection device that detects a movement of a user's hand in the operation space in a non-contact manner. When the detection device detects a movement of a hand of a user in the operation space, the movement of the hand is displayed on the non-contact interface screen in the form of a translucent image or a pointer. This prevents a hand of a user from blocking the user's view of the non-contact interface screen when the user operates the screen, allowing the user to perform input while checking information on the screen.

Description

Technical Field

[0006]

[0001] The present invention relates to a non-contact input device that can perform input operations without hindrance while viewing a screen.

Background Art

[0002] The pandemic caused by the recent novel coronavirus has made people realize how vulnerable the 21st century society, with increasing population density and the era of high-speed mass transportation, is to infectious diseases. In such a situation, everything around us seems to be a source of infection, and the living situation becomes extremely cramped. In fact, during the spread of the infection, it is necessary to suspect everything.

[0003] It is important to take measures such as wearing masks, keeping a sufficient distance from others, and avoiding crowds. However, in order to lead a necessary social life, contact with the minimum infection route cannot be avoided anyway. Minimizing this risk is crucial for preventing infection.

[0004] For example, at bank ATMs, supermarket self-checkouts, company receptions, hotel check-ins, etc., an unspecified number of users operate the same touch panel. Such touch panels can be a potential infection route. To avoid this risk, it is necessary to disinfect the touch panel every time the user changes, but such measures are quite time-consuming and difficult to perform thoroughly.

[0005] In recent years, in such a situation, a technology that can non-contactedly indicate an aerial image formed in the air using a special optical element has attracted attention. By using this technology, a completely non-contact interface can be realized with the same usability as a conventional touch panel.

[0006] For example, Patent Document 1 describes a procedure in which, because touching a computer mouse or other pointing device during surgery poses hygienic problems, the surgeon operates a mouse on a non-contact remote pointer control device displayed using aerial imaging technology. An example of such a special optical element is the optical imaging device described in Patent Document 2. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2018-147054 [Patent Document 2] International Publication No. 2009 / 131128 [Overview of the project] [Problems that the invention aims to solve]

[0008] One typical application of aerial imaging technology is a non-contact input device, as described in Patent Document 1, but it has some drawbacks compared to using a real mouse. Specifically, when using a real hand to operate an aerial display, the real hand inevitably obstructs the display, making it difficult to see. In contrast, with normal mouse operation for computer work, only a small pointer is displayed on the screen, so the display is almost entirely visible. Therefore, operating an aerial display can be confusing compared to using a normal mouse.

[0009] Therefore, the object of the present invention is to provide a non-contact input device that does not obstruct the display when input is performed using the hand. [Means for solving the problem]

[0010] To solve the above problems, a non-contact input device according to one aspect of the present invention is: The aerial display comprises a display device and an optical plate that focuses the image of the display device onto an airborne region as a focused image; an operating space provided at a location separate from the airborne region; and a detection device that non-contactually detects the movement of a user's hand in the operating space. The focused image formed in the air by the display device and the optical plate includes a non-contact interface screen as an input means and an image of the user's hand detected by the detection device. The image of the hand is displayed either as an outline or semi-transparently so as not to obstruct the non-contact interface screen. The user can see the elements constituting the non-contact interface screen through this outline-only or semi-transparent image of the hand, and can perform input operations on the non-contact interface screen via the image of the hand displayed in the focused image by moving their hand in the operating space. It is characterized by the following:

[0011] Furthermore, in one embodiment, the input operation of the non-contact interface screen can also be performed by the user directly touching the non-contact interface of the focused image.

[0012] Furthermore, in one embodiment, the operating space is located below the contactless interface screen.

[0013] Furthermore, in one embodiment, The aerial display comprises a display device and an optical plate that focuses the image of the display device onto an airborne region as a focused image; an operating space provided at a location separate from the airborne region; and a detection device that non-contactually detects the movement of a user's hand in the operating space. The focused image formed in the air by the display device and the optical plate includes an image of a non-contact interface screen as an input means and an image of a pointer indicating the position of the user's fingertip detected by the detection device. The user can perform input operations on the non-contact interface screen via the pointer displayed on the focused image by moving their hand in the operating space. . [Effects of the Invention]

[0014] According to the contactless input device of the present invention, when operating a contactless interface screen, the user's hands do not obstruct the screen, allowing them to input information while checking the information on the screen, and enabling smooth input operations. [Brief explanation of the drawing]

[0015] [Figure 1] Figure 1 is a perspective view showing a non-contact input device 1 in use according to an embodiment of the present invention. [Figure 2] Figure 2 is a side view showing the contactless input device 1 in use according to an embodiment of the present invention. [Figure 3] Figure 3 shows an example of a contactless interface screen displayed by the contactless input device 1 according to an embodiment of the present invention. [Figure 4] Figure 4 shows another example of a contactless interface screen displayed by the contactless input device 1 according to an embodiment of the present invention. [Figure 5] Figure 5 illustrates the operation of the contactless interface screen shown in Figure 4. [Figure 6] Figure 6 shows yet another example of a contactless interface screen displayed by the contactless input device 1 according to an embodiment of the present invention. [Figure 7] Figure 7 illustrates the operation of the contactless interface screen shown in Figure 6. [Figure 8]FIG. 8 is a perspective view showing a non-contact input device 5 in a usage state according to another implementation of an embodiment of the present invention. [Figure 9] FIG. 9 is a side view showing the folded state of the non-contact input device 1 according to an embodiment of the present invention. [Figure 10] FIG. 10 is a front view showing the folded state of the non-contact input device 1 according to an embodiment of the present invention. [Figure 11] FIG. 11 is a rear view showing the folded state of the non-contact input device 1 according to an embodiment of the present invention. [Figure 12] FIG. 12 is a view showing a dark screen used in the non-contact input device 1 according to an embodiment of the present invention. [Figure 13] FIG. 13 is a perspective view showing a state where the dark screen of FIG. 12 is attached to the non-contact input device 1 according to an embodiment of the present invention.

MODE FOR CARRYING OUT THE INVENTION

[0016] Hereinafter, embodiments of the non-contact input device according to the present invention will be described with reference to the accompanying drawings. FIG. 1 is a perspective view showing the non-contact input device 1 in a usage state according to an embodiment of the present invention. FIG. 2 is a side view showing the non-contact input device 1 in a usage state according to an embodiment of the present invention.

[0017] As shown in these figures, the non-contact input device 1 includes a support substrate 10 on which an information processing device that performs overall signal processing is mounted, a liquid crystal display 20 connected to the support substrate 10 via a hinge portion 12, and an optical plate 30 connected to the liquid crystal display 20 via a hinge portion 22.

[0018] The information processing device mounted on the support substrate 10 is substantially a small computer and is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a storage device that stores various programs and data, an input / output interface, and the like. As the input / output interface, for example, a USB port 14 or a wireless LAN such as WIFI is mounted.

[0019] The liquid crystal display 20 has a display surface 24 on the side facing the optical plate 30. That is, in the operational state of the non-contact input device 1 shown in Figures 1 and 2, the display surface 24 is angled upwards. Based on the display data transmitted from the support substrate 10, a non-contact interface is displayed on the display surface 24. The non-contact interface here consists of components such as buttons, icons, and menus, and the user can input information into the system by operating these components.

[0020] The optical plate 30 has its incident surface 31 facing downwards towards the display surface 24 of the liquid crystal display 20, and its exit surface 33 facing directly upwards. The image on the display surface 24 of the liquid crystal display 20 is then focused again as an aerial display 4 at the same distance on the opposite side, forming the same image as before.

[0021] As such an optical plate 30, for example, an optical imaging element described in Japanese Patent Application Publication No. 2011-175297 can be used. This optical imaging element is realized by arranging a large number of mutually orthogonal planar light reflecting parts at a constant pitch. In addition, a structure such as a two-sided corner reflector, in which a reflective surface is formed on the side surface of a square-shaped hole, as described in Japanese Patent No. 4900618, may also be used.

[0022] Furthermore, an infrared LED 32 and a pair of infrared cameras 34 are provided on the optical plate 30 near the hinge portion 22. These infrared LED 32 and infrared cameras 34 constitute an operation detection unit that detects user operations on the aerial display 4. The detection range of the operation detection unit is defined by the emission angle of the infrared LED 32 and the field of view of the infrared cameras 34.

[0023] The support substrate 10, the liquid crystal display 20, and the optical plate 30 are connected by hinge portions 12 and 22, respectively, but are fixed in the operational state of the non-contact input device 1 shown in Figures 1 and 2 by a pair of left and right locking rods 40.

[0024] In other words, the locking rod 40 is pivotally supported by a shaft 42 located in the center of the side surface of the liquid crystal display 20, and is capable of engaging with the engaging recess 16 of the support substrate 10 and the engaging recess 36 of the optical plate 30. When both ends of the locking rod 40 are engaged with the engaging recess 16 of the support substrate 10 and the engaging recess 36 of the optical plate 30, the optical plate 30 is supported horizontally, and the liquid crystal display 20 is fixed at an angle of approximately 45 degrees downward relative to the optical plate 30. In this operating state, the liquid crystal display 20 and the support substrate 10 are also fixed at an angle of approximately 45 degrees.

[0025] It is important that the angle between the liquid crystal display 20 and the optical plate 30 is approximately 45 degrees. At this angle, the light from the liquid crystal display 20 is imaged onto the aerial display 4 under optimal conditions. Also, the liquid crystal display 20 and the support substrate 10 are at approximately 45 degrees, and at this angle the optical plate 30 is horizontal. More specifically, it is desirable that the angle be in the range of roughly 40 to 50 degrees.

[0026] In this state, when an operation interface is displayed on the liquid crystal display 20, it is projected onto the opposite side of the optical plate 30 as a non-contact interface screen 4. When a gesture is made to this non-contact interface screen 4, such as by touching it with a finger, the operation is detected by an operation detection unit consisting of an infrared LED 32 and an infrared camera 34, and a corresponding operation signal is sent to the information processing device on the support substrate 10, where predetermined processing is performed. Components of the operation interface include, for example, controls such as buttons, checkboxes, and drop-down menus.

[0027] Furthermore, in the non-contact input device 1 according to the present invention, an image sensor 48 is provided in the upper center of the back surface of the liquid crystal display 20, and an image sensor 49 is provided in the inner center of the hinge portion 12. The image sensor 48 captures the space between the support substrate 10 and the liquid crystal display 20 (hereinafter referred to as the operating space S) from above, and the image sensor 49 captures the operating space S between the support substrate 10 and the liquid crystal display 20 from the front. These image sensors 48 and 49 detect the movement of a hand H inserted into the operating space S between the support substrate 10 and the liquid crystal display 20.

[0028] The hand H detected by these image sensors 48 and 49 is displayed on the display surface 24 of the liquid crystal display 20 and on the aerial display 4 as a focused image through its optical plate 30. However, in order not to obstruct the non-contact interface displayed on the aerial display 4, only the outline H' of the hand H is displayed or it is displayed semi-transparently. Movement of the hand H enables operation of the non-contact interface on the aerial display 4 using this hand H'.

[0029] As a method for detecting the hand H in the operating space S, for example, the positions of the five fingertips can be identified, and from there, the outline H' or semi-transparent image of the hand H can be created on the aerial display 4. Alternatively, instead of an outline or semi-transparent image, it may be effective to display only pointers indicating the fingertips on the aerial display 4, depending on the application. In this case, the simplest method is to identify only the position of the index finger and display one pointer on the aerial display 4. Alternatively, the positions of the index finger and thumb can be identified and two pointers can be displayed on the aerial display 4, or all five fingers can be displayed as five pointers on the aerial display 4. By moving these pointers, the non-contact interface of the aerial display 4 can be operated.

[0030] Therefore, there are two methods for inputting to the contactless input device 1 in this embodiment. One is the conventional method of directly operating the contactless interface displayed on the aerial display 4, and the other is a method of operating the contactless interface via the hand H' displayed on the aerial display 4 by moving the hand H in the operating space between the support substrate 10 and the liquid crystal display 20, rather than directly operating the contactless interface on the aerial display 4 with the hand.

[0031] In other words, if the infrared camera 34 detects a hand directly operating the non-contact interface displayed on the aerial display 4, it accepts input according to that operation. On the other hand, if the image sensors 48 and 49 detect a hand H, they display the corresponding hand H' movement on the aerial display 4 according to that operation and accept input via this hand H'. Alternatively, a pointer is displayed instead of hand H', and input is accepted via this pointer.

[0032] For example, in the case of a contactless interface screen such as a simple button selection as shown in Figure 3, it is easy to operate the air display 4 directly with the hand. On the other hand, in the case of a contactless interface where multiple selections are made while reading text, as shown in Figure 4 (in this case, checkboxes), the user inserts their hand into the operating space S between the support substrate 10 and the liquid crystal display 20.

[0033] The image sensors 48 and 49 then detect the hand H and display a pointer P on the aerial display 4 corresponding to the position of the index finger. The user moves their index finger while looking at the pointer P on the aerial display 4 to align the pointer P with one of the checkboxes. If the pointer P is held in this position for a certain period of time (e.g., 3 seconds or more), the checkbox is selected. If the pointer P is held in this position for a certain period of time (e.g., 3 seconds or more) with the selected checkbox, the checkbox is deselected (see Figure 5). With this method, the text of the checkboxes is not hidden by the user's hand, allowing the user to select while confirming the content.

[0034] In another example shown in Figure 6, when the image sensors 48 and 49 detect a hand H, they identify the positions of the index finger and thumb and display two pointers P1 and P2 on the contactless interface screen of the aerial display 4. These two pointers P1 and P2 move according to the movement of the index finger and thumb.

[0035] The user, while looking at pointers P1 and P2 displayed on the aerial display 4, moves their index finger and thumb to pinch icon I on the screen with pointers P1 and P2, and drops icon I onto the print icon PT (see Figure 7). Here, icon I represents an image file. When this movement is detected, the system prints the corresponding image using the printer (not shown). If icon I is dropped onto the display icon DP instead of the print icon PT, the system displays the corresponding image.

[0036] As described above, in the embodiment of the contactless input device according to the present invention, both a method of directly operating the aerial display 4 with the hand and a method of indirectly operating the contactless interface screen by inserting the hand into the operating space S provided below are available.

[0037] However, the infrared LED 32 and infrared camera 34 may be omitted, and only the method of indirectly operating the non-contact interface screen by inserting a hand into the operating space S may be enabled. In this case, since there is only one method of operation, the user will not be confused. Also, in this case, in terms of realizing a non-contact interface, there is no necessity to implement an aerial display. Therefore, the non-contact input device according to the present invention can be realized even if the system is built using only a liquid crystal display without using an aerial display device including an optical plate 30.

[0038] Figure 8 shows a specific example of a contactless input device realized without using an aerial display device. Here, the elements corresponding to each element of the contactless input device in Figure 1 are given the same reference numerals in this contactless input device 5, and their descriptions will not be repeated. Here, the optical plate 30 in Figure 1 is replaced by a liquid crystal display 60, and a support substrate 65 is provided in the position of the liquid crystal display 20 in Figure 1. However, the liquid crystal display 60 is supported in a position that is easy for the user to see, with the display raised to the position of the aerial display 4 in Figure 1.

[0039] Similarly, in the non-contact input device 5, an image sensor 48 is provided in the upper center of the back surface of the support substrate 65, and an image sensor 49 is provided in the inner center of the hinge portion 12. The image sensor 48 captures the operating space S from above, and the image sensor 49 captures the operating space S between the support substrate 10 and the support substrate 65 from the front. These image sensors 48 and 49 detect the movement of the hand H inserted into the operating space S.

[0040] The hand H detected by these image sensors 48 and 49 is displayed on the liquid crystal display 60. However, as in the example shown in Figure 1, in order not to obstruct the non-contact interface displayed on the liquid crystal display 60, only the outline H' of the hand H is displayed, or it is displayed semi-transparently, or a pointer corresponding to the position of the fingertip is displayed. The movement of the hand H enables operation of the non-contact interface of the air display 4 using this hand H', as shown in Figures 4 to 7.

[0041] Furthermore, this non-contact input device 1 is highly portable. Specifically, when carrying it, both ends of the locking rod 40 are detached from the engaging recess 16 of the support substrate 10 and the engaging recess 36 of the optical plate 30, and the support substrate 10, liquid crystal display 20, and optical plate 30 are rotated at the hinge portion 12 and hinge portion 22, and stacked horizontally to form a compact structure as shown in the side view of Figure 9, the front view of Figure 10, and the rear view of Figure 11.

[0042] Therefore, the contactless input device according to the embodiment of the present invention is portable and can be folded into a small size for easy carrying. For example, this contactless input device can be carried in a bag, and online payments can be made contactlessly at the destination.

[0043] As shown in Figure 1, a speaker 18 can also be provided on the support board 10. For example, if some operation is performed on the non-contact interface screen 4 and detected, the user can be notified with a click sound or other means. Guidance voices can also be played for unfamiliar users. Specifically, on an interface screen like the one in Figure 3, voice guidance such as "Please touch the button on the aerial display directly" can be played, and on an interface screen like the one in Figure 4, voice guidance such as "Move your index finger on the support board below to move the cursor on the screen to the box to the left of the item you want to select" can be played.

[0044] Furthermore, in order to achieve a clearer aerial display 4, it is desirable that external light not be added to the light from the liquid crystal display 20. For this purpose, it is advisable to use a blackout curtain 44 as shown in Figure 12. This blackout curtain 44 is intended to cover the space between the liquid crystal display 20 and the optical plate 30, and consists of a left shielding portion 44L, a central shielding portion 44C, and a right shielding portion 44R.

[0045] The left shielding portion 44L covers the left open area between the liquid crystal display 20 and the optical plate 30, the central shielding portion 44C covers the central open area, i.e., the back area, between the liquid crystal display 20 and the optical plate 30, and the right shielding portion 44R covers the right open area between the liquid crystal display 20 and the optical plate 30.

[0046] A hook-and-loop fastener 46 is provided on the edge of the blackout curtain 44. Although not directly shown in the figure, a hook-and-loop fastener is also provided on the corresponding surface of the non-contact input device 1. The hook-and-loop fasteners on the blackout curtain 44 and the non-contact input device 1 are male and female on one side, and as shown in Figure 13, they can be bonded together to completely cover the space between the liquid crystal display 20 and the optical plate 30. This prevents external light from being added to the light from the liquid crystal display 20, resulting in a clearer aerial display 4.

[0047] Although the blackout curtain 44 is shown here to be attached when in use, it may also be permanently fixed to the liquid crystal display 20 and optical plate 30 without being attached or removed. In this case, when folding the non-contact input device 1, it should be folded between the liquid crystal display 20 and optical plate 30. One way to fold it is, for example, by folding it in a zigzag pattern like a fan. [Industrial applicability]

[0048] According to the present invention, a contactless input device can be realized that allows users to operate a contactless interface screen without their hands obstructing the screen, enabling them to input information while checking the information on the screen, and facilitating smooth input operations.

[0049] Although the present invention has been described in detail above with reference to examples, it will be clear to those skilled in the art that the present invention is not limited to the examples described herein. The apparatus of the present invention can be implemented in modified and altered forms without departing from the spirit and scope of the invention as defined by the claims. Therefore, the description herein is for illustrative purposes only and is not intended to be restrictive in any way to the present invention.

[0050] For example, while a liquid crystal display is used in the above embodiment, the present invention is not limited thereto, and organic EL displays, electronic paper, etc., may also be used. Furthermore, while a locking rod is used as a support structure for fixing the display and optical panel, the present invention is not limited thereto. For example, a locking mechanism may be implemented in the hinge part itself, or a shielding plate may be used instead of a blackout curtain, and this shielding plate may be used as a support structure for fixing the display and optical panel. [Explanation of symbols]

[0051] 1, 5, 6 Contactless Input Devices 4. Aerial Display 10 Support substrate 12. Hinge section 14 ports 16 Engaging recess 18 speakers 20 LCD displays 22 Hinge section 24 Display surface 30 Optical Plates 31 Incidence plane 32 Infrared LEDs 33. Ejection surface 34 Infrared Cameras 36 Engaging recess 40 Locking Rod 42 axes 44 Blackout Curtain 44C Central shielding part 44L Left side shielding part 44R Right side shielding part 46 hook-and-loop fasteners 48, 49 Image sensor

Claims

1. An aerial display comprising a display device and an optical plate that focuses the image of the display device onto an aerial region as a focused image, An operating space provided at a location separate from the aforementioned aerial region, The system includes a detection device that non-contactually detects the user's hand movements in the aforementioned operating space. The non-contact input device is characterized in that the focused image formed in the air by the display device and the optical plate includes a non-contact interface screen as an input means and an image of the user's hand detected by the detection device, the image of the hand is displayed as an outline only or semi-transparently so as not to obstruct the non-contact interface screen, the user can see the elements constituting the non-contact interface screen through this outline-only or semi-transparent displayed image of the hand, and input operations on the non-contact interface screen can be performed by moving the hand in the operating space via the image of the hand displayed in the focused image.

2. The non-contact input device according to Claim 1, characterized in that the input operation of the non-contact interface screen can also be performed by the user directly touching the non-contact interface of the focused image.

3. The contactless input device according to claim 1, characterized in that the operating space is located below the contactless interface screen.

4. An aerial display comprising a display device and an optical plate that focuses the image of the display device onto an aerial region as a focused image, An operating space provided at a location separate from the aforementioned aerial region, The system includes a detection device that non-contactually detects the user's hand movements in the aforementioned operating space. The contactless input device is characterized in that the focused image formed in the air by the display device and the optical plate includes an image of a contactless interface screen as an input means and an image of a pointer indicating the position of the user's fingertip detected by the detection device, and the user can perform input operations on the contactless interface screen via the pointer displayed in the focused image by moving their hand in the operating space.