Non-contact input apparatus and input assistance application

The non-contact input device uses an electrostatic sensor to measure two-dimensional positions and distances, allowing three-dimensional input in devices with two-dimensional applications, thereby expanding interaction capabilities.

JP2024162833APending Publication Date: 2024-11-21STELLARLINK CORP +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023078754
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Conventional input terminal devices require dedicated applications to detect three-dimensional positions, limiting their functionality in devices equipped with applications that only handle two-dimensional positions.

Method used

A non-contact input device that measures a two-dimensional position and distance from an operation surface using an electrostatic sensor, and a control device that outputs a second pointer and click events based on these measurements, enabling three-dimensional input in devices with two-dimensional capable applications.

Benefits of technology

Enables non-contact three-dimensional input in devices originally designed for two-dimensional inputs, enhancing user interaction without the need for dedicated three-dimensional detection capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024162833000001_ABST
    Figure 2024162833000001_ABST
Patent Text Reader

Abstract

To provide a non-contact input apparatus that enables non-contact input through three-dimensional action in an apparatus equipped with an application program that can support only a two-dimensional position.SOLUTION: A non-contact input apparatus includes: an operation surface; a detection unit that measures a two-dimensional position of an object that performs operation on the operation surface in a non-contact manner and a distance from the operation surface to the object; a display that is disposed on the back side of the operation surface; and a control apparatus that has an input assistance application and an OS which uses a planar GUI. The input assistance application outputs, to the OS, an instruction so that the OS overlays a second pointer, which is different from a first pointer displayed on the display, on the display in accordance with the two-dimensional position and the distance. When it is determined that the operation of the object has been decided on the basis of the two-dimensional position and the distance, the input assistance application outputs, to the OS, a click event and the two-dimensional position at which the operation was decided.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a non-touch input device and an input assistance application. [Background technology]

[0002] Conventionally, there is an input terminal device for inputting a user's operation via a position input object, which includes a position detection unit that detects the position of the position input object operated by the user in a non-contact manner, a display unit that displays a pointer based on the position detected by the position detection unit, and an operation processing control unit that executes a corresponding operation process based on the position detected by the position detection unit. The operation processing control unit has a plurality of operation modes related to the execution of the operation process, one of which includes a state in which no operation process is executed other than the movement of the pointer according to the position of the position input object, and the operation processing control unit switches between the plurality of operation modes when the user performs a specific operation via the position input object (see, for example, Patent Document 1). It is also known that the size of the pointer is changed according to the position of the hand approaching the operation screen, which position the hand is facing, and the distance between the hand and the operation screen, based on the three-dimensional spatial position of the hand (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2007-064771 [Patent Document 2] JP 2010-244422 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional input terminal devices use a dedicated application program capable of detecting three-dimensional positions in order to detect the three-dimensional position of a position input object (object) in a non-contact manner. For this reason, conventional input terminal devices cannot be realized with a device equipped with an application program capable of handling only two-dimensional positions.

[0005] Therefore, an object of the present invention is to provide a non-contact input device and an input assistance application that enable non-contact input with three-dimensional movements in a device equipped with an application program that can only handle two-dimensional positions. [Means for solving the problem]

[0006] A non-contact input device according to an embodiment of the present disclosure includes an operation surface, a detection unit that measures a two-dimensional position of an object performing an operation on the operation surface in a non-contact manner and a distance from the operation surface to the object, a display arranged on the rear side of the operation surface, and a control device having an OS that uses a planar GUI and an input assistance application, wherein the input assistance application outputs a command to the OS to display a second pointer, different from a first pointer displayed by the OS on the display, as an overlay on the display in accordance with the two-dimensional position and the distance, and outputs the two-dimensional position and a click event to the OS based on the two-dimensional position and the distance. Effect of the Invention

[0007] It is possible to provide a non-contact input device and an input assistance application that enable non-contact input with three-dimensional movements in a device equipped with an application program that can only handle two-dimensional positions. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an example of a configuration of a non-contact input device according to an embodiment. [Diagram 2] 1 is a diagram illustrating an example of a configuration of an electrostatic sensor, a control device, and the like of a non-contact input device according to an embodiment; [Diagram 3] 11A and 11B are diagrams illustrating an example of four regions according to the distance in the Z direction from the operation surface of the non-contact input device according to the embodiment. [Figure 4] 2 is a diagram illustrating an OS, an input assistance application, and a predetermined application program of a control unit of the non-contact input device according to the embodiment. FIG. [Figure 5A] 10A and 10B are diagrams illustrating an index of the positional accuracy of an object in the non-contact input device according to the embodiment. [Figure 5B] 10A and 10B are diagrams illustrating an index of the positional accuracy of an object in the non-contact input device according to the embodiment. [Figure 5C] 10A and 10B are diagrams illustrating an index of the positional accuracy of an object in the non-contact input device according to the embodiment. [Figure 6A] FIG. 4 is a diagram illustrating an example of a display on a display of the non-contact input device according to the embodiment. [Figure 6B] FIG. 4 is a diagram illustrating an example of a display on a display of the non-contact input device according to the embodiment. [Figure 6C] FIG. 4 is a diagram illustrating an example of a display on a display of the non-contact input device according to the embodiment. [Figure 6D] FIG. 4 is a diagram illustrating an example of a display on a display of the non-contact input device according to the embodiment. [Figure 6E] FIG. 4 is a diagram illustrating an example of a display on a display of the non-contact input device according to the embodiment. [Figure 6F] FIG. 4 is a diagram illustrating an example of a display on a display of the non-contact input device according to the embodiment. [Figure 7A] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 7B] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 7C] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 7D] 11A and 11B are diagrams illustrating an example of a modified example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 8A] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 8B] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 8C] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 8D] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 8E] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 8F] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 9A] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 9B] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 9C] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 9D] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 9E] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 9F] FIG. 11 is a diagram illustrating an example of a process executed by an input assistance application of the non-contact input device according to the embodiment. [Figure 10] FIG. 13 illustrates an example of a setting screen. [Figure 11] FIG. 2 is a diagram illustrating an example of a folder structure. [Figure 12] 13 is a flowchart illustrating an example of a setting process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment to which a non-contact input device and an input assistance application according to the present disclosure are applied will be described.

[0010] <Embodiment> Fig. 1 is a diagram showing an example of the configuration of a non-contact input device 100 according to an embodiment. Fig. 1 shows a state in which a display 110 displays an input image.

[0011] Fig. 2 is a diagram showing an example of the configuration of the electrostatic sensor 120, the control device 130, etc. of the non-contact input device 100. Fig. 3 is a diagram showing four regions according to the distance from the operation surface 105A of the non-contact input device 100 in the Z direction.

[0012] In the following, the XYZ coordinate system is defined and described. The X axis is the first axis, the Y axis is the second axis, and the Z axis is the third axis. The direction parallel to the X axis (X direction), the direction parallel to the Y axis (Y direction), and the direction parallel to the Z axis (Z direction) are mutually perpendicular. In the following, the -Z direction is described as the direction approaching the electrostatic sensor 120, and the +Z direction is described as the direction away from the electrostatic sensor 120. In addition, a planar view refers to a view in the XY plane. In the following, the length, width, thickness, etc. of each part may be exaggerated to make the configuration easier to understand.

[0013] The non-contact input device 100 may be, for example, a tablet-type input device that is placed in a store or facility and used by an unspecified number of users, or an input section of an ATM (Automatic Teller Machine), or may be an input section of a cooking appliance that needs to be kept clean.

[0014] <Overall Configuration of Non-Contact Input Device 100> The non-contact input device 100 includes a housing 101, a top panel 105, a display 110, an electrostatic sensor 120, an input sensor circuit 125A, an image display circuit 125B, and a control device 130. The input sensor circuit 125A calculates XYZ coordinates from the measurement results of the electrostatic sensor 120. Although the control device 130 (see FIG. 2) is omitted in FIG. 1, the control device 130 is provided, as an example, below the display 110 and the electrostatic sensor 120 inside the housing 101. The non-contact input device 100 includes the electrostatic sensor 120 and the control device 130 shown in FIG. 2.

[0015] <Chassis 101 and top panel 105> The housing 101 is a case made of resin, metal, or the like that houses the display 110, the electrostatic sensor 120, and the control device 130. The display 110 is disposed below the transparent electrostatic sensor 120, and is visible through an operation surface 105A that is the upper surface of a transparent top panel 105 provided in an opening at the top of the housing 101. The electrostatic sensor 120 may be retrofitted to an existing display 110. The display and the control device 130 may be integrated as in a tablet computer. The display and the control device 130 may be separate as in a desktop computer.

[0016] <Four Areas of the Non-Contact Input Device 100 and Operation Method> The non-contact input device 100 can be operated in both a state where a pointer such as a user's hand is not in contact with the operation surface 105A and a state where a pointer such as a user's hand is in contact with the operation surface 105A. Here, the non-contact input device 100 will be described together with the modes in which an input assistance application is executed. The input assistance application has four modes: a decision mode, a selection mode, a proximity mode, and a standby mode.

[0017] The non-contact input device 100 is an input device that is operated by a user performing a pointing operation. The pointing operation is an operation performed by holding a finger substantially perpendicular to the operation surface 105A. The number of fingers used in the pointing operation may be multiple, but it is preferable to use one finger.

[0018] When performing such a pointing operation, if the finger is not approximately perpendicular to the operation surface 105A, the entire palm approaches the operation surface 105A, making it difficult to measure the position of the fingertip FT.

[0019] In the following, unless otherwise specified, a case will be described where a user performs a pointing operation with a fingertip FT as a pointer. In addition, in the following, performing an operation (proximity operation, selection operation, decision operation, or touch operation) with the fingertip FT will be simply expressed as an operation (proximity operation, selection operation, decision operation, or touch operation) with the fingertip FT.

[0020] There are three types of operation methods for the non-contact input device 100: a proximity operation, a selection operation, and a decision operation.

[0021] In order to distinguish between the four operation methods, the non-contact input device 100 uses four areas shown in Fig. 3. The four areas are, in order from the operation surface 105A side, a decision area, a selection area, a proximity area, and a standby area.

[0022] <Decision area, selection area, proximity area, and waiting area> The decision area is a first area whose Z-direction distance from the operation surface 105A is shorter than Z1 (for example, 2 cm), and is an area in which the input assistance application sets the mode of the input assistance application to the decision mode. Z1 is an example of a first threshold value. When the input assistance application determines that the fingertip FT is located within the decision area, a decision operation is performed. Note that the decision area also includes a Z-direction distance of 0 cm (contact) from the operation surface 105A. Note that the relationship between the Z-direction distance and the capacitance is affected by individual differences and the installation environment. However, if the operator and the usage environment are the same, the capacitance is inversely proportional to the Z-direction distance. For this reason, if it is sufficient to measure the relative distance to the same operator, the capacitance can be regarded as the distance. Note that the decision area and the decision operation will be described in detail later.

[0023] Instead of the distance in the Z direction, the input assistance application uses a value (Z value) proportional to the capacitance between the fingertip FT and the electrostatic sensor 120. The input sensor circuit 125A calculates the position (XY coordinates) of the fingertip FT facing the operation surface 105A and a value (Z value) proportional to the capacitance from the operation surface 105A to the fingertip FT, based on the capacitance at multiple intersections between the multiple sensor electrodes 121X and the multiple sensor electrodes 121Y.

[0024] The decision area is an area where the Z value is greater than Cz1 (for example, 60). Cz1 is a value proportional to the capacitance corresponding to the distance Z1. Cz1 is an example of a first threshold value. In this embodiment, the case where the fingertip FT touches the operation surface 105A is also included in the decision area. However, it is also possible to determine from the Z value that the fingertip FT has touched the operation surface 105A. For example, it is possible to design the Z value to be 100 when the fingertip FT lightly touches the operation surface 105A. If the Z value is greater than 100, it may be determined that the fingertip FT is in the contact area, and the operation may be performed in contact mode.

[0025] The selection area is a second area that is at a distance from the operation surface 105A that is equal to or greater than Z1 and shorter than Z2 (e.g., 4 cm) that is longer than Z1, and is an area in which the input assistance application sets the mode of the input assistance application to the selection mode. Z2 is an example of a second threshold. When the input assistance application determines that the fingertip FT is located within the selection area, a selection operation is performed. The selection operation will be described in detail later.

[0026] The selected region is a region where the Z value is greater than Cz2 (e.g., 40) and less than or equal to Cz1. Cz2 is a value smaller than Cz1. Cz2 and Cz1 are values ​​proportional to the capacitances corresponding to the distances Z2 and Z1, respectively. The capacitance Cz2 is an example of a second threshold value.

[0027] The proximity area is a third area whose distance from the operation surface 105A is equal to or greater than Z2 and shorter than Z3 (e.g., 7 cm) that is longer than Z2, and is an area in which the input assistance application sets the mode of the input assistance application to a proximity mode. Z3 is an example of a third threshold. When the input assistance application determines that the fingertip FT is located within the proximity area, a proximity operation is performed. The proximity area is the area farthest from the operation surface 105A among areas in which the XY coordinates of the target object can be calculated. Details of the proximity operation will be described later.

[0028] The proximity region is a region where the Z value is greater than Cz3 (e.g., 10) and less than or equal to Cz2. Cz3 is a value smaller than Cz2. Cz3 and Cz2 are values ​​proportional to the capacitances corresponding to the distances Z3 and Z2, respectively. Cz3 is an example of a third threshold value.

[0029] The standby area is a fourth area whose distance from the operation surface 105A is longer than Z3, and is an area where the input assistance application sets the mode of the input assistance application to standby mode. The standby area is an area whose Z value is equal to or less than Cz3. The capacitance value measured when the fingertip FT is not present around the operation surface 105A is equal to the reference value. The Z value is calibrated to 0 when the fingertip FT is not present around the operation surface 105A. The input assistance API may turn off the power of the display in standby mode.

[0030] <Proximity operation, selection operation, decision operation, and touch operation> The proximity operation is an operation of bringing the fingertip FT close to the operation surface 105A of the non-contact input device 100 without touching it, and is an operation for switching the display on the display 110 from the standby mode display to the proximity mode display.

[0031] The selection operation is an operation in which, from the state in which the proximity operation has been performed, the fingertip FT is brought even closer to the operation surface 105A of the non-contact input device 100 without touching it, to select a GUI button displayed on the display 110.

[0032] The decision operation is an operation of issuing a click event by bringing the fingertip FT closer to the operation surface 105A of the non-contact input device 100 without touching it from the state where the selection operation has been performed. When a click event is issued on the GUI button, the OS 140 and the application program 160 decide the operation input. The decision operation is to perform an operation input without contact, and to operate the non-contact input device 100 without touching the operation surface 105A with the fingertip FT. The operation input performed by the non-contact selection operation and decision operation may be called a hover input or a touchless input. Note that even if the fingertip FT touches the operation surface 105A of the non-contact input device 100, it may be treated as a decision mode. Alternatively, when the fingertip FT touches the operation surface 105A of the non-contact input device 100, the input may be immediately confirmed. Alternatively, when the fingertip FT touches the operation surface 105A of the non-contact input device 100, a warning screen may be displayed.

[0033] <Display 110> The display 110 is a liquid crystal display, an organic EL (Electroluminescence) display, or the like. The display 110 is a display for realizing a GUI (Graphic User Interface). The display 110 displays an image (desktop screen, etc.: not shown) provided by a graphical shell of the OS, a second pointer 112 overlaid by the input support application 150, and an image 115 of the application software. The input support application 150 overlays the second pointer 112 by placing the second pointer 112 at the top of the Z order (the order closest to the viewpoint). The image 115 of the application software includes an image of a GUI button 110A. The GUI button 110A is an operation unit, and is shaped like a push button, for example. The application software is designed to be operated with a two-dimensional position input device such as a mouse.

[0034] FIG. 1 shows the screen of an ordering terminal of a restaurant as an example. The ordering screen shows a total of 17 GUI buttons 110A, including eight GUI buttons 110A for menus and nine GUI buttons 110A in the form of a numeric keypad for numbers or the like. The 17 GUI buttons 110A are arranged in three or four rows in the Y direction and five rows in the X direction. The rows extend in the X direction, and Y extends in the column direction. Note that the GUI buttons 110A are not limited to those on the screen of an ordering terminal of a restaurant, and may be other applications as long as they are an operation screen for a touch panel.

[0035] <Electrostatic sensor 120, input sensor circuit 125A, image display circuit 125B> The electrostatic sensor 120 is disposed on top of the display 110, and has a plurality of sensor electrodes 121X extending in the X direction and a plurality of sensor electrodes 121Y extending in the Y direction, as shown in FIG. 2. An input sensor circuit 125A is integrally provided on the electrostatic sensor 120. An image display circuit 125B is integrally provided on the display 110. The input sensor circuit 125A is connected between the wirings 122X, 122Y and the control device 130. The image display circuit 125B is connected between the display 110 and the control device 130.

[0036] The sensor electrodes 121X, 121Y are connected to the control device 130 via wirings 122X, 122Y and an input sensor circuit 125A. Such an electrostatic sensor 120 can be formed by forming a transparent conductive film such as ITO (Indium Tin Oxide) on the surface of transparent glass and patterning it into the sensor electrodes 121X, 121Y and wirings 122X, 122Y. The electrostatic capacitance detected by the electrostatic sensor 120 is input to the control device 130.

[0037] 2 shows a plurality of sensor electrodes 121X and a plurality of sensor electrodes 121Y. Note that it is preferable that the distance between the sensor electrodes 121X and the distance between the sensor electrodes 121Y are narrower than the distance between the GUI buttons 110A. In other words, it is preferable to use electrostatic sensors 120 that correspond to the distance between the GUI buttons 110A.

[0038] The input sensor circuit 125A is mounted on a wiring board. The input sensor circuit 125A is provided between the wirings 122X, 122Y and the control device 130, and performs AD (Analog to Digital) conversion of the electrostatic capacitance of the electrostatic sensor 120 acquired by sequentially selecting the multiple wirings 122X and the multiple wirings 122Y. The input sensor circuit 125A calculates the XY coordinates of the fingertip FT and a Z value proportional to the electrostatic capacitance value between the operation surface 105A and the fingertip FT from the electrostatic capacitance value of each wiring. The input sensor circuit 125A is capable of outputting XY in the same format as a touch-operated digitizer, and is also capable of outputting XYZ in a unique format. The input assistance application 150 sends a command to stop the XY output in the digitizer format to the input sensor circuit 125A via a coordinate device driver. The input assistance application 150 performs processing based on the XYZ output. The image display circuit 125B is provided between the display 110 and the control device 130, and displays an image on the display 110 in accordance with image data sent from the control device 130.

[0039] <Control device 130> The control device 130 has a control unit 131 and a memory 132. The control device 130 is realized by a computer including a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), an input / output interface, an internal bus, and the like. The control unit 131 represents the functions of a program executed by the control device 130 as a functional block. The memory 132 represents the memory of the control device 130 functionally.

[0040] The control unit 131 controls the operation of the non-contact input device 100. The control unit 131 receives the XY coordinates and Z value input from the input sensor circuit 125A. The control unit 131 controls the display of the image on the display 110 through the image display circuit 125B. The control unit 131 outputs a command corresponding to the operation content determined by the determination operation of the fingertip FT. For example, if the non-contact input device 100 is an ordering terminal of a restaurant, it transmits the name and number of dishes ordered by the customer to an order management terminal in the kitchen.

[0041] The control unit 131 has an OS that uses a planar GUI, a device driver for coordinates, a device driver for display, an input support application, and a predetermined application program. Here, Fig. 4 is used to explain the OS, the input support application, and the predetermined application program. Fig. 4 is a diagram for explaining the OS, the input support application, and the predetermined application program of the control unit 131.

[0042] 4 shows the OS 140, the input assistance application 150, and the application program 160 that the control unit 131 has. The OS 140 is an OS (Operating System) that uses a planar GUI, such as Windows (registered trademark) or Android (registered trademark). An OS that uses a planar GUI does not provide a UI (User Interface) that is suitable for three-dimensional input. The application program 160 is an application program for an ordering terminal of a restaurant, or the like.

[0043] The OS 140 has a coordinate device driver 141 and a display device driver 142. The coordinate device driver 141 passes the XY coordinates and Z value input from the electrostatic sensor 120 to the input support application 150. When the XY coordinates and a click event are input from the input support application 150, the OS 140 causes the application program 160 to execute a command corresponding to the button on the input XY coordinates. In addition, the OS 140 passes data of an image to be displayed, the XY position of the image to be displayed, and the Z order of the image to be displayed (order of distance from the viewpoint) to the display device driver 142. The Z order of the second pointer 112 is at the top by the input support application 150.

[0044] The display device driver 142 performs control to display the second pointer 112 at the XY coordinates input from the input support application 150, and control to display an image according to a command input from the application program 160 when a confirmation operation is performed. The OS treats the second pointer 112 simply as an image.

[0045] The input assistance application 150 is an application that provides input assistance to a user who uses the non-contact input device 100. The input assistance application 150 may perform a process of calculating an index of the positional accuracy of the fingertip FT. The input assistance application 150 performs a process of causing the OS 140 to display an image of the second pointer 112, which is different from the first pointer 111 that the OS 140 displays on the display 110, based on the XY coordinates of the fingertip FT and the index of the positional accuracy. Furthermore, when a confirmation operation is performed, the input assistance application 150 outputs the XY coordinates at the time of the confirmation operation to the OS 140 as the position of the mouse pointer, and then issues a click event. The OS 140 notifies the application program 160 that the GUI button corresponding to the position of the mouse pointer at the time the click event is issued has been pressed. Other processes performed by the input assistance application 150 and details of each process will be described later.

[0046] Application program 160 is designed to be operated with a normal mouse (that outputs two-dimensional coordinates) or a touch panel. When application program 160 is notified by OS 140 that a click event has been issued on a GUI button, it performs processing according to the GUI button that was clicked.

[0047] The display 110, the control device 130, the OS 140, the display device driver 142, and the application program 160 may be existing devices that do not support three-dimensional input (non-contact input). By fixing the electrostatic sensor 120 capable of non-contact input to the existing display 110 and installing the coordinate device driver 141 and the input assistance application 150 in the control device 130, the existing device becomes a non-contact input device.

[0048] <Calculation of XY coordinates and Z value by input sensor circuit 125A> The input sensor circuit 125A scans the sensor electrodes 121X row by row and scans the sensor electrodes 121Y column by column, and converts the capacitances at the intersections of the sensor electrodes 121X and 121Y into digital values. The control unit 131 counts the change in the output of the digital value of the capacitance and calculates a difference value ΔAD at each intersection. The difference value ΔAD is a count value of the change in the output of the input sensor circuit 125A relative to a reference value. The reference value is a value proportional to the capacitance of each intersection of the sensor electrodes 121X and 121Y when there is no object such as a fingertip FT around the sensor electrodes 121X and 121Y. The input sensor circuit 125A calculates the XY coordinates on the display facing the fingertip FT from the difference value at each intersection, and selects the maximum ΔAD of the difference values ​​ΔAD at each intersection as a Z value. The Z value is a value proportional to the capacitance between the fingertip FT and the operation surface. The XY coordinates and the Z value may be calculated by the control unit 131.

[0049] It is also possible to increase the resolution by using an interpolation method, based on the distance between the sensor electrodes 121X and the distance between the sensor electrodes 121Y. In this case, the distance between the sensor electrodes 121X and the distance between the sensor electrodes 121Y may be greater than the distance between the GUI buttons 110A. The coordinate device driver 141 may calculate the XY coordinates and the Z value. When the coordinate device driver 141 calculates the XY coordinates and the Z value, the hardware that performs the calculation is the control unit 131.

[0050] <Index of target position accuracy> The input assistance application 150 obtains an index of the positional accuracy of an object when displaying the second pointer 112 on the display 110. The object is an object for which the XY coordinates and the distance in the Z direction from the operation surface 105A are obtained based on the electrostatic capacitance with the electrostatic sensor 120, and is a part of the body such as the hand or fingertip of the user of the non-contact input device 100. Here, the index of the positional accuracy of the object will be described.

[0051] 5A and 5B are diagrams for explaining an index of the positional accuracy of an object. The non-contact input device 100 uses the amount of variation in the XY coordinates (two-dimensional position) of the fingertip FT as an index of the positional accuracy of the object.

[0052] 5A shows an example of the distribution of capacitance in the X direction measured at times t1 and t2. Assume that the X coordinate of the fingertip FT is calculated as X1 at time t1, and the X coordinate of the fingertip FT is calculated as X2 at time t2.

[0053] Even if the fingertip FT is not moving, the X coordinate fluctuates due to noise. There is a difference between the X coordinate X1 at time t1 and the X coordinate X2 at time t2, and this difference in coordinates appears as the amount of fluctuation in the coordinates. The same is true for the Y coordinate.

[0054] The non-contact input device 100 uses the amount of fluctuation of coordinates measured within a certain time as an index of positional accuracy. More specifically, the standard deviation of the measured coordinates is used as an index of positional accuracy. For example, if the second pointer 112 is a circle with a radius of 1σ of the standard deviation, the probability that a finger is inside the second pointer 112 is about 68%, and if the second pointer 112 is a circle with a radius of 2σ, the probability that a finger is inside the second pointer 112 is about 95%. In this way, by expressing the positional accuracy by the size of the pointer, it is possible to intuitively feed back the positional accuracy to the user.

[0055] Fig. 5B shows an example of the capacitance distribution in the X direction when a fingertip FT is held upright on the operation surface 105A and is gradually brought closer to the operation surface 105A. The left fingertip FT in Fig. 5B is the furthest from the operation surface 105A, and the right fingertip FT in Fig. 5B is the closest to the operation surface 105A. The double-headed arrows in Fig. 5B indicate the range in which the capacitance is detected. Note that Fig. 5B shows the capacitance distribution in the X direction, but as explained in Fig. 5C, the capacitance distribution in the Y direction is similar due to symmetry.

[0056] When the fingertip FT is far from the operation surface 105A (left side), the electrostatic sensor 120 is affected by the fingertip FT over a wide range, resulting in a flat capacitance distribution. If the magnitude of the noise is the distance between the two dotted lines, the range that may be detected as the X coordinate of the fingertip FT is the range with a larger capacitance than the lower dotted line. In other words, the range in the X direction where the fingertip FT may be detected is the wide range shown by the left-right arrow.

[0057] Also, when the fingertip FT is close to the operation surface 105A (on the right side), the electrostatic sensor 120 is affected by the fingertip FT in a narrow range, resulting in a steep distribution of capacitance. If the magnitude of the noise is the interval between the two dotted lines, the range that may be detected as the X coordinate of the fingertip FT is the range with a larger capacitance than the lower dotted line. In other words, the range in the X direction where the fingertip FT may be detected is the narrow range shown by the left-right arrow.

[0058] Furthermore, when the distance of the fingertip FT from the operation surface 105A is intermediate as shown in the center of FIG. 5B, the distribution of the capacitance in the X direction is intermediate between the gradual distribution on the left side and the steep distribution on the right side.

[0059] In this way, the range in which the measured XY coordinates fluctuate due to noise is larger when the fingertip FT is farther from the operation surface 105A (left side) than when the fingertip FT is closer to the operation surface 105A (right side). The larger the area in which the capacitance is detected, the larger the amount of fluctuation in the coordinates becomes.

[0060] Fig. 5C shows an example of the distribution of capacitance in the X direction when the fingertip FT is operated upright on the operation surface 105A (left side) and when the fingertip FT is operated lying down on the operation surface 105A (right side). The double-headed arrows indicate the range in which the capacitance is detected. Note that although Fig. 5C shows the distribution of capacitance in the X direction, the distribution of capacitance in the Y direction is similar.

[0061] As shown on the left side of Fig. 5C, when the fingertip FT is operated in an upright position, the capacitance distribution becomes steeper and the range in which the fingertip FT is detected becomes narrower. As shown on the right side of Fig. 5C, when the fingertip FT is laid down and the palm is close to the operation surface 105A, the capacitance distribution becomes flatter and the range in which the fingertip FT is detected becomes wider.

[0062] In this way, the amount of variation in the measured XY coordinates due to noise is greater when the fingertip FT is operated lying down, as shown on the right side of Figure 5C, than when the fingertip FT is operated upright, as shown on the left side of Figure 5C.

[0063] As described above with reference to FIGS. 5A, 5B, and 5C, the amount of fluctuation of the measured XY coordinates due to noise varies depending on the distance from the fingertip FT to the operation surface 105A, the orientation of the fingertip FT, and the like. Furthermore, the amount of fluctuation also varies depending on the magnitude of the noise. Furthermore, the amount of fluctuation also increases when the fingertip FT is actually moved. If the amount of fluctuation increases for any reason, the measurement accuracy is poor, so the second pointer 112 is displayed larger. Furthermore, if the amount of fluctuation decreases, the second pointer 112 is displayed smaller. This allows the position accuracy to be represented by the size of the second pointer, and the position accuracy can be intuitively fed back to the user.

[0064] 5C, the amount of variation in the measured XY coordinates is larger when the fingertip FT is laid down and the palm is close to the operation surface 105A than when the fingertip FT is held upright. Therefore, when the amount of variation in the XY coordinates is larger than a predetermined threshold, the non-contact input device 100 may determine that the fingertip FT is not held upright near the operation surface 105A and display a message encouraging the user to hold the fingertip FT upright near the operation surface 105A.

[0065] <Calculating the XY coordinates of the object (calculating the moving average)> 5A to 5C, since the XY coordinates calculated by the input sensor circuit 125A have fluctuations, the input assistance application 150 calculates the moving average of the X coordinates acquired at multiple time points and the moving average of the Y coordinates acquired at multiple time points as the XY coordinates of the fingertip FT. The multiple time points are the most recent time points for calculating the moving average, and the interval between the time points is several milliseconds.

[0066] The input assistance application 150 outputs a command to the OS 140 to display an image of the second pointer at the XY coordinates calculated as the moving average.

[0067] The XY coordinates of the fingertip FT detected by the non-contact input device 100 using the electrostatic sensor 120 represent, as an example, the XY coordinates of the position with the largest capacitance in the area where the fingertip FT exists. The center of gravity of a figure consisting of a plurality of points whose difference from the largest capacitance is equal to or less than a threshold value may be regarded as the XY coordinates of the fingertip FT. Also, the X coordinate (or Y coordinate) of the apex obtained by fitting the position with the largest capacitance and the capacitance of three points adjacent to it in the X direction (or Y direction) to a quadratic curve may be regarded as the X (Y) coordinate of the fingertip FT. Also, the capacitance is inversely proportional to the distance. Since the capacitance and the distance correspond one-to-one, the capacitance can be used instead of the distance. Also, in the non-contact input device 100 of the present invention, the unit of the capacitance does not need to be expressed in farads [F]. In the present invention, a value proportional to the capacitance is called a Z value. The input sensor circuit 125A is designed to set the Z value to about 100 when the fingertip FT is in contact with the operation surface 105A, and to set the Z value to about 0 when the fingertip FT is not near the operation surface 105A.

[0068] In the proximity mode or the standby mode, the input assistance application 150 may stop calculating the moving average of the XY coordinates or the amount of fluctuation of the XY coordinates. Also, in the standby mode, the power supply of the display may be turned off.

[0069] <Calculation of Index of Position Accuracy of Object by Input Assistance Application 150> In order to indicate the magnitude of error due to the amount of fluctuation in the XY coordinates measured by the non-contact input device 100, the input assistance application 150 calculates the standard deviation of the XY coordinates as an index of the positional accuracy of the target object. A method for calculating the standard deviation of the XY coordinates will be described later with reference to FIG. 9D.

[0070] <Display example of non-contact input device 100> 6A to 6F are diagrams showing an example of a display on the display 110 of the non-contact input device 100. In Fig. 6A and Fig. 6B, the display 110 and the operation surface 105A are shown overlapping each other. The display on the display 110 can be viewed by the user through the operation surface 105A. Fig. 6C to Fig. 6F show a part of an image displayed on the display 110.

[0071] Fig. 6A shows the display of the display 110 in the standby mode. When the user's fingertip FT is not near the operation surface 105A, as shown in Fig. 6A, messages such as "Touchless operation is possible" and "Please place your finger on the screen" are displayed. This allows the user to guide the fingertip FT to the operation surface 105A even when using the non-contact input device 100 for the first time, leading to smooth use thereafter.

[0072] Fig. 6B shows an example of a message in the proximity mode. In Fig. 6B, messages such as "The pointer will shrink depending on the position accuracy" and "If the pointer is large, move your finger closer or hold it up" are displayed. When the user brings the fingertip FT closer to the operation surface 105A, a second pointer 112 is displayed on the display 110. Note that the pointer is a figure, a symbol, or the like that indicates the current input position on the screen (image) displayed on the display 110.

[0073] The second pointer 112 is a figure that the input support application 150 causes the OS 140 to display. The input support application 150 hides the first pointer (mouse pointer) 111 prepared by the OS. The second pointer 112 may be a circular or elliptical figure, in addition to the annular figure shown in FIG. 6B. The input support application 150 increases the radius of the second pointer 112 as the amount of variation in the XY coordinates, which is an index of positional accuracy, increases, and decreases the radius of the second pointer 112 as the amount of variation in the XY coordinates decreases. In other words, the input support application 150 causes the display 110 via the OS 140 to display the second pointer 112 with a larger radius as the amount of variation in the XY coordinates increases, and causes the display 110 via the OS 140 to display the second pointer 112 with a smaller radius as the amount of variation in the XY coordinates decreases.

[0074] Here, a more specific operation when the user brings the fingertip FT closer to the operation surface 105A to select the GUI button for the number 6 in the selection mode will be described with reference to Fig. 6C. In Fig. 6C, the first pointer 111 is hidden, and only the second pointer 112 is shown. In the selection mode, the second pointer 112 is displayed in a circular shape.

[0075] 6C, the input assistance application 150 causes the OS 140 to display the second pointer 112 whose radius increases as the amount of variation in the XY coordinates increases. For example, when the fingertip FT is farther from the operation surface 105A, the amount of variation in the XY coordinates increases.

[0076] 6C, the input assistance application 150 causes the OS 140 to display the second pointer 112 with a smaller radius as the amount of variation in the XY coordinates decreases. For example, when the fingertip FT is closer to the operation surface 105A, the amount of variation in the XY coordinates decreases.

[0077] That is, when operating the GUI button for the number 6, if the fingertip FT is far from the operation surface 105A, the second pointer 112 with a large radius is displayed as shown on the left side of FIG. 6C. Then, if the fingertip FT is brought closer to the operation surface 105A, the radius of the second pointer 112 becomes smaller, and the second pointer 112 with a small radius is displayed as shown on the right side of FIG. 6C. In this way, if the fingertip FT is far from the operation surface 105A and the precision of the XY coordinates is low, the radius of the second pointer 112 is set to be large, and if the fingertip FT is close to the operation surface 105A and the precision of the XY coordinates is high, the radius of the second pointer 112 is set to be small. In this way, the non-contact input device 100 can intuitively feed back the position precision to the user.

[0078] FIG. 6D illustrates an operation when the user holds the fingertip FT close to the operation surface 105A to confirm the operation input of the GUI button of the number 6 in the confirmation mode. The second pointer 112A has a different display color from the second pointer 112 shown in FIG. 6C. For this reason, in FIG. 6C, the second pointer 112A is shown by a double line. The input assistance application 150 intuitively indicates that the input mode has changed by changing the circular shape of the second pointer 112 to the arc of the second pointer 112A. Furthermore, the input assistance application 150 can emphasize the change of the input mode by changing the color of the second pointer 112 from red (112) to yellow (112A). The color of the second pointer is not limited to being changed from red to yellow. The color of the second pointer (112, 112A) may be changeable on a setting screen described later. Instead of changing the color, the brightness may be changed, or both the color and the brightness may be changed.

[0079] FIG. 6D shows how the images change over time, starting from the top left, followed by the arrows, then to the top right, bottom left, and bottom right.

[0080] In the decision mode, first, as shown in the upper left, an arc-shaped second pointer 112A is displayed, and when the fingertip FT is held close to the operation surface 105A, the arc extends as shown in the upper right, and further extends as shown in the lower left. When the first predetermined time T1 has elapsed since the fingertip FT was stopped on the operation surface 105A, the second pointer 112A becomes a ring. When the second pointer 112A becomes a ring, the operation content is decided. That is, the operation content is decided by continuing to point the number 6 with the fingertip FT until the second pointer 112A becomes a ring. In this way, the non-contact input device 100 allows the user to perform a non-contact click operation instead of an operation of touching the operation surface 105A.

[0081] When the operation content is determined, the input assistance application 150 outputs a click event. The input assistance application 150 issues the same event as when a mouse is clicked by inputting an operation with the fingertip FT without contact. The input assistance application 150 outputs the XY coordinates of the fingertip FT to the OS 140 and outputs a click event. In addition, the input assistance application 150 may cause the OS 140 to play a click event output sound. Such output of the XY coordinates without contact and output of the click event corresponds to the operation of touching the GUI button 110A on the operation surface 105A.

[0082] When the input assistance application 150 outputs a click event, it displays a circular second pointer 112B having a smaller radius than the annulus for a predetermined time (for example, about 1 to 2 seconds) as shown in the lower right of FIG. 6D.

[0083] The circular second pointer 112B is displayed for the user's confirmation at the XY coordinates (moving average) of the fingertip FT when the operation input is confirmed. As an example, the second pointer 112B is displayed for a predetermined time (about 1 to 2 seconds) and becomes invisible when the predetermined time has elapsed. The second pointer 112B may have a display color different from that of the second pointers 112 and 112A. The input assistance application 150 intuitively indicates that the operation input has been confirmed by changing the second pointer 112B to a small circle immediately after the central angle of the second pointer 112A (arc) becomes 360 degrees. Furthermore, by changing the color, it is possible to emphasize that the operation input has been confirmed. Instead of changing the color, the brightness may be changed, or both the color and the brightness may be changed.

[0084] 6C and 6D show an example in which the first pointer 111 of the OS 140 is erased, but as shown in FIG. 6E, both the first pointer 111 (mouse pointer) of the OS 140 and the second pointer 112 (pointer for a non-contact input device) by the input assistance application 150 may be displayed. When both the first pointer 111 and the second pointer 112 are displayed, the input assistance application 150 continues to output XY coordinates to the OS 140 in a selection mode and a decision mode, which will be described later. The first pointer 111 performs the same function as a reticle that indicates the position of the XY coordinates with high precision.

[0085] <Display function menu> Fig. 6F shows the function menu 112C. In the selection mode, when the fingertip FT is brought close to the operation surface 105A and the radius of the second pointer 112 is reduced as shown on the right side of Fig. 6C, and the XY coordinates of the fingertip FT are stationary for a predetermined time, the input assistance application 150 causes the OS 140 to display the function menu 112C. This predetermined time is an example of a third predetermined time. Fig. 6E shows, as an example, the function menu 112C from which a click mode, a drag mode, and a pinch-in can be selected.

[0086] <Flowchart> 7A to 7C, 8A to 8F, and 9A to 9F are diagrams showing an example of processing executed by the input assistance application 150 of the non-contact input device 100. Fig. 7A to 7C show an example of main processing, Fig. 7D shows an example of a modified example of the processing shown in Fig. 7C, and Fig. 8A to 8F and 9A to 9F show an example of subroutine processing.

[0087] When the input assistance application 150 starts processing (see FIG. 7A), it sets initial values ​​(step S1). Step S1 is a subroutine, the details of which will be described later with reference to FIG. 8A, and sets initial values ​​such as the number of data Nma for calculating a moving average.

[0088] The input assistance application 150 outputs a command to erase the first pointer 111 to the OS 140 (step S2). This causes the first pointer 111 to be erased from the display 110. Note that the first pointer 111 (mouse pointer) may be displayed without performing the process of step S2 (see FIG. 6E). In this case, both the first pointer 111 and the second pointer 112 will be displayed. In this case, the input assistance application 150 continues to output the XY coordinates of the fingertip FT to the OS. Since the position of the first pointer 111 and the center of the second pointer 112 coincide with each other, the center of the second pointer 112 becomes easy to see even if the circle in which it is a large circle is displayed.

[0089] The input assistance application 150 calculates the XY coordinates of the fingertip FT (step S3). Step S3 is a subroutine, the details of which will be described later, but it calculates a moving average of the XY coordinates of the fingertip FT using the XY coordinates of the fingertip FT calculated at multiple points in time.

[0090] The input support application 150 determines whether the mode of the input support application 150 is the decision mode (step S4). For ease of understanding, the case where the mode of the input support application 150 is the decision mode will be described first.

[0091] When the input support application 150 determines that the mode of the input support application 150 is the decision mode (S4: YES), the input support application 150 determines whether the Z value is greater than Cz1 (step S5). If the Z value is greater than Cz1, the input support application 150 considers that the distance from the operation surface 105A to the fingertip FT is shorter than the first threshold value Z1. The input support application 150 determines whether the fingertip FT is located within the decision area from the Z value.

[0092] If the input assistance application 150 determines that the Z value is greater than Cz1 (S5: YES), it sets SelectionOffTime and ProximityOffTime to the current time (step S6). SelectionOffTime is updated to the time when it was determined that the Z value was greater than Cz1. Thus, SelectionOffTime indicates the last time when it was determined that the Z value was greater than Cz1. As will be described later, ProximityOffTime is also updated when it is determined that the Z value is greater than Cz2. Thus, ProximityOffTime indicates the last time when it was determined that the Z value was greater than Cz2.

[0093] The input assistance application 150 determines whether DecisionTH is 0 (S7). DecisionTH is a value that can be set by the user. If DecisionTH is 0, the input is confirmed immediately upon entering decision mode without displaying a circular arc cursor, which will be described later. On the other hand, if DecisionTH is other than 0, the input is confirmed after displaying a circular arc cursor, which will be described later.

[0094] When the input assistance application 150 determines that DecisionTH is not 0 (S7: No), it outputs to the OS 140 a command to display the second pointer 112A in an arc shape at the XY coordinates (moving average: Xave, Yave) calculated in step S3 (step S8). The command output to the OS 140 in step S8 is a command to display an arc with a radius of a predetermined value (fixed value) centered on the XY coordinates (moving average). The arc is an arc with a moving end extending clockwise from the fixed end, the fixed end is located at a position corresponding to 12 o'clock on a clock, and the moving end extends clockwise from 12 o'clock. The input assistance application 150 calculates the angle of the arc between the fixed end and the moving end, for example, using the following formula (1). Angle of the arc = 360 degrees × (current time - DecisionTime) / DecisionTH (1)

[0095] Here, DecisionTime is the time when drawing of the arc starts, and DecisionTH is the time required from when drawing of the arc starts until a click event is output. A click event is an event that is performed when the operation content is decided by an operation input with the fingertip FT. The operation content is decided when the second pointer 112A changes from an arc to a ring. When the operation content is decided, the input assistance application 150 outputs a click event. The user can decide the operation content by bringing the fingertip FT close to the operation surface 105A within the decision area and holding the fingertip FT until the second pointer 112A extends to become a ring.

[0096] The input assistance application 150 determines whether the current time-DecisionTime>DecisionTH is satisfied (step S9). That is, it determines whether the time elapsed from the time (DecisionTime) when the mode transitioned to the decision mode exceeds DecisionTH. This is to determine whether the operation content has been decided, and to determine whether to output a click event when the operation content has been decided. The time elapsed from the time (DecisionTime) when the mode transitioned to the decision mode is an example of a first predetermined time.

[0097] If the input assistance application 150 determines that the current time-DecisionTime>DecisionTH is not true (S9: NO), the input assistance application 150 returns the flow to step S3. The processes of steps S3 to S8 are repeated, so that the arc extends.

[0098] When the input assistance application 150 determines that the current time-DecisionTime>DecisionTH is true (S9: YES), the input assistance application 150 proceeds with the process to output a click event. When the current time-DecisionTime>DecisionTH is true, the arc cursor becomes a ring.

[0099] The input assistance application 150 erases the second pointer 112A (step S10).

[0100] The input assistance application 150 outputs the XY coordinates (moving averages) calculated in step S3 to the OS 140, and then outputs a click event to the OS 140. Furthermore, the input assistance application 150 causes the OS 140 to play a click event output sound (step S11). This causes the click event output sound to be output from the speaker. Furthermore, when the XY coordinates overlap with the GUI button 110A of the application program 160 and a click event is issued, the application program 160 executes the process indicated by the GUI button 110A.

[0101] If the input assistance application 150 determines that DecisionTH is 0 (S7: Yes), it executes the process of step S11 described above. That is, if the input assistance application 150 determines that DecisionTH is 0 (S7: Yes) or if it determines that the current time-DecisionTime>DecisionTH is true (S9: YES), it outputs the XY coordinates and the click event to the OS 140 and causes the OS 140 to play the click event output sound.

[0102] The input assistance application 150 causes the OS 140 to display the second pointer 112B centered on the XY coordinates (moving average) calculated in step S3 for a predetermined period of time (step S12).

[0103] The center of second pointer 112B is the XY coordinates (moving average) calculated in step S3. The predetermined time is, for example, about 1 to 2 seconds, and when the predetermined time has elapsed, second pointer 112B becomes invisible.

[0104] The input assistance application 150 judges whether ReClickTH=0 is satisfied (step S13). ReClickTH is the time (click re-output time) until the second pointer 112A starts to be displayed again when the fingertip FT continues to be positioned within the decision area after a click event is output. The user of the non-contact input device 100 can set ReClickTH for the input assistance application 150 through the setting screen displayed on the display 110.

[0105] When the input assistance application 150 determines that ReClickTH=0 is true (S13: YES), it calculates the XY coordinates of the fingertip FT (step S14). Step S14 is a subroutine similar to step S3, and the details of which will be described later, but it calculates a moving average of the XY coordinates of the fingertip FT using the XY coordinates of the fingertip FT calculated at multiple points in time.

[0106] The input support application 150 determines whether the Z value is greater than Cz1 (step S15). After the operation content is determined, the input support application 150 determines whether the fingertip FT continues to be located within the determination area.

[0107] If the input assistance application 150 determines that the Z value is not greater than Cz1 (S15: NO), it sets the mode to selection mode (step S16). Then, the flow returns to step S3. If the Z value ΔAD is not greater than Cz1, this is the case when the fingertip FT has moved from the decision area to the selection area, the proximity area, or the like. When the fingertip FT has moved from the decision area to the selection area, the proximity area, or the like, the process of step S14 is provided to enable re-input.

[0108] If the input assistance application 150 determines in step S15 that the Z value is greater than Cz1 (S15: YES), it causes the OS 140 to display the second pointer 112B at the XY coordinate (moving average) calculated in step S14 (step S17). The size of the circular second pointer 112B is a predetermined value.

[0109] When the click re-output time (ReClickTH) is set to 0, the input assistance application 150 continues to display the second pointer 112B at the XY coordinate position of the fingertip FT without re-outputting the click event unless the mode is changed from the decision mode to another mode.

[0110] When the input support application 150 finishes the process of step S17, it returns the flow to S14. If the fingertip FT continues to be within the decision area after the operation content is decided as YES in step S7 or YES in step S9 and the second pointer 112B is displayed in step S12, the input support application 150 displays the second pointer 112B in step S17 and then returns the flow to step S14. That is, the second pointer 112B continues to be displayed while updating the position of the second pointer 112B. After the operation content has been decided once, in order to decide the next operation content, it is sufficient to move the fingertip FT once outside the decision area.

[0111] <s5:no> Furthermore, when the input assistance application 150 determines in step S5 that the Z value is not greater than Cz1 (S5: NO), it determines whether or not the relationship Current time - SelectionOffTime > SelectionOffTH is established (step S18). That is, when the fingertip FT is continuously positioned outside the decision area, it determines whether or not the elapsed time from the time (SelectionOffTime) when it was last determined that the fingertip FT was positioned within the decision area exceeds SelectionOffTH. This elapsed time is an example of a fourth predetermined time.

[0112] SelectionOffTH is the time required to transition from the decision mode to the selection mode when the fingertip FT is moved to the selection area. The user of the non-contact input device 100 can set SelectionOffTH for the input assistance application 150 through a setting screen displayed on the display 110. Note that the time when the fingertip FT was last determined to be located within the decision area is the most recent time when the fingertip FT was determined to be located within the decision area.

[0113] When the input assistance application 150 determines that the current time-SelectionOffTime>SelectionOffTH is not satisfied (S18: NO), it determines whether the Z value is greater than Cz2 (step S19). When the Z value is greater than Cz2, the input assistance application 150 considers that the fingertip FT is staying in the first area. When Z is greater than Cz2 (S19: YES), the flow returns to step S7. As a result, the operation in the decision mode continues. The Z value does not use a moving average, and a value smaller than the actual value may be measured due to noise. In other words, even if the fingertip FT is located in the decision area, there are cases where Z>Cz1 is not satisfied (S5: NO). Therefore, even if the Z value is not greater than Cz1, if the Z value is greater than Cz2, the second pointer 112A continues to be drawn for a while (during SelectionOffTH). On the other hand, if the Z value is not greater than Cz2 (S19: NO), the input assistance application 150 sets the selection mode and sets the current time to SelectionTime (step S20). If the Z value is not greater than Cz2, the fingertip FT is considered not to be in the decision area.

[0114] If the input support application 150 determines that the current time-SelectionOffTime>SelectionOffTH is satisfied (S18: YES), it sets the mode of the input support application 150 to the selection mode and sets SelectionTime to the current time (step S20). If the state in which the Z value is not larger than Cz1 (step S5) continues longer than SelectionOffTH, it is determined that the fingertip FT is not in the decision area.

[0115] The input assistance application 150 causes the OS 140 to hide the second pointer 112A (step S21).

[0116] The input support application 150 causes the OS 140 to display the second pointer 112 (step S22). The second pointer 112 is the second pointer for the selection mode shown in FIG. 6C. Before causing the OS 140 to display the second pointer 112, the input support application 150 causes the second pointer 112A to be hidden. The center of the second pointer 112 is the XY coordinates (moving average) of the fingertip FT, and the radius of the second pointer 112 is a radius according to the amount of fluctuation of the XY coordinates. When causing the OS 140 to display the second pointer 112 in step S22, the input support application 150 outputs the position of the second pointer 112 according to the XY coordinates (moving average) of the fingertip FT to the OS 140. In addition, the input support application 150 sets the size of the second pointer 112 to a size according to the amount of fluctuation of the XY coordinates.

[0117] The input support application 150 executes a process of determining whether or not the function menu needs to be displayed (step S23). The process of determining whether or not the function menu needs to be displayed is a subroutine process, and will be described in detail later. After completing the process of step S23, the input support application 150 returns the flow to step S3.

[0118] <s12:no> If the input assistance application 150 determines in step S13 that ReClickTH=0 is not true (S13: NO), it sets ClickTime to the current time (step S24). ClickTime is the time when the click event is output.

[0119] The input assistance application 150 determines whether or not the relationship: current time-ClickTime>ReClickTH is satisfied (step S25).

[0120] If the current time-ClickTime>ReClickTH is established (S25: YES), the input assistance application 150 sets DecisionTime to the current time (step S26). Then, the flow returns to step S3. DecisionTime becomes the time when the flow returns to step S3. When the fingertip FT is held in the decision area, a gradually longer arc is drawn again. The time from when a click event is output once until the drawing of the second pointer 112A (arc) starts again is the "predetermined time" in step S12 plus ReClickTH. It takes DecisionTimeTH from when the drawing of the second pointer 112A (arc) starts until the second pointer 112A becomes a ring. A click event is output every time the time obtained by adding the "predetermined time" in step S12, ReClickTH, and DecisionTimeTH has elapsed. In other words, the time obtained by adding the "predetermined time" in step S12, ReClickTH, and DecisionTimeTH becomes the second predetermined time.

[0121] If the input assistance application 150 determines in step S25 that the current time-ClickTime>ReClickTH is not satisfied (S25: NO), it calculates the XY coordinates of the fingertip FT (step S27). Step S27 is a subroutine similar to step S3, and the details of which will be described later, but the moving average of the XY coordinates of the fingertip FT is calculated using the XY coordinates of the fingertip FT calculated at multiple points in time.

[0122] The input assistance application 150 determines whether the Z value is greater than Cz1 (step S28) in order to determine whether the fingertip FT is located within the decision area.

[0123] If the input support application 150 determines that the Z value is greater than Cz1 (S28: YES), it displays the second pointer 112B at a position centered on the XY coordinates of the fingertip FT (step S29). If the input support application 150 remains in the decision mode, it displays the second pointer 112B on the OS 140 until the click re-output time (ReClickTH) has elapsed.

[0124] If the input support application 150 determines in step S27 that the Z value is not greater than Cz1 (S28: NO), it sets the mode of the input support application 150 to the selection mode and sets the current time in SelectionTime (step S30). When the fingertip FT moves from the decision area to the selection area, the mode immediately transitions to the selection mode. A user who wants to perform the next operation input as soon as possible can switch to the selection mode by slightly removing the fingertip FT from the operation surface 105A. When the input support application 150 finishes the process of step S30, it returns the flow to step S3.

[0125] <s4:no> Furthermore, when the input support application 150 determines in step S4 that the mode is not the decision mode (S4: NO), it determines whether the Z value is greater than Cz1 (step S4A). That is, the input support application 150 determines whether the fingertip FT is located within the decision area. If the Z value is greater than Cz1, the fingertip FT is located within the decision area.

[0126] When the input assistance application 150 determines that the Z value is greater than Cz1 (S4A: YES), it sets SelectionOffTime and ProximityOffTime to the current time (step S5A). SelectionOffTime is updated to the time when it is determined that the Z value is greater than Cz1, so SelectionOffTime indicates the last time when the Z value was determined to be greater than Cz1. As will be described later, ProximityOffTime is also updated when it is determined that the Z value is greater than Cz2. Therefore, ProximityOffTime indicates the last time when the Z value was determined to be greater than Cz2.

[0127] The input support application 150 sets DecisionTime to the current time (step S6A). When the time during which the decision mode has continued reaches DecisionTH, the input support application 150 outputs a click event. Since it is determined in step S4 that the decision mode is not in effect and then in step S4A that the Z value is greater than Cz1, the time at which step S6A is executed is the time at which the decision mode is in effect. In other words, the DecisionTime set in step S6A is the time at which the decision mode is in effect.

[0128] The input support application 150 sets the mode of the input support application 150 to the decision mode (step S7A). Furthermore, the input support application 150 erases the second pointer 112 for the selection mode (step S8A). After completing the process of step S8A, the flow proceeds to step S7.

[0129] The flow proceeds from step S4: NO to step S4A: YES when the fingertip FT enters the decision area from outside the decision area. In this way, when the fingertip FT approaches the operation surface 105A, the mode is immediately changed. On the other hand, when the fingertip FT moves away from the operation surface 105A, the mode is not immediately changed. As described above, when the Z value becomes small (when the state in which the fingertip FT moves away from the operation surface 105A is measured), the mode is not immediately changed, thereby stabilizing the operation. On the other hand, when the moving average is not used for the Z value and the fingertip FT approaches the operation surface 105A, the mode is immediately changed to the decision mode, thereby realizing a quick operation. Note that when the decision mode is entered due to the influence of noise, the fingertip FT is not in the decision area but is near the decision area (selection area). For this reason, the user does not feel that it is a malfunction. It is possible to prevent both a decrease in reaction speed and erroneous input.

[0130] <s4a:no> Furthermore, if the input support application 150 determines in step S4A that the Z value is not greater than Cz1 (S4A: NO), it determines whether the mode of the input support application 150 is the selection mode (step S4B).

[0131] In the selection mode, the input support application 150 determines whether the Z value is greater than Cz2 (step S5B). That is, the input support application 150 determines whether the fingertip FT is located within the selection area.

[0132] When the input assistance application 150 determines that the Z value is greater than Cz2 (S5B: YES), it sets ProximityOffTime to the current time (step S6B). ProximityOffTime is the time when the fingertip FT was last determined to be located within the selection area when the fingertip FT was continuously located within the selection area. When the fingertip FT is located within the selection area, ProximityOffTime is updated, so that ProximityOffTime becomes the most recent time when the fingertip FT was determined to be located within the selection area.

[0133] The input assistance application 150 executes a second pointer radius setting process to set the radius of the second pointer 112 according to the amount of variation in the X and Y coordinates (step S7B). The second pointer radius setting process is a subroutine process and will be described in detail later, but in the selection mode, the radius of the second pointer 112 (see FIG. 6C) is set according to the amount of variation in the X and Y coordinates. After completing the process of step S7B, the input assistance application 150 advances the flow to step S21 and causes the second pointer 112 to be displayed on the OS 140.

[0134] <s5b:no> When the input assistance application 150 determines in step S5B that the Z value is not greater than Cz2 (S5B: NO), it determines whether or not the current time-ProximityOffTime>ProximityOffTH is satisfied (step S8B). That is, when the fingertip FT is continuously positioned within the selection area, it determines whether or not the elapsed time from the time (ProximityOffTime) when it was last determined that the fingertip FT was positioned within the selection area exceeds ProximityOffTH. ProximityOffTH is the time until the transition from the selection mode to the proximity mode. The user of the non-contact input device 100 can set ProximityOffTH for the input assistance application 150 through the setting screen displayed on the display 110.

[0135] When the input assistance application 150 determines that the current time-ProximityOffTime>ProximityOffTH is not satisfied (S8B: NO), it executes a second pointer radius setting process to set the radius of the second pointer 112 according to the amount of fluctuation in the XY coordinates (step S9B). The second pointer radius setting process is a subroutine process, so the details will be described later, but in the selection mode, the radius of the second pointer 112 (see FIG. 6C) is set according to the amount of fluctuation in the XY coordinates. When the input assistance application 150 finishes the process of step S9B, the flow proceeds to step S21, and the second pointer 112 is displayed on the OS 140. The Z value does not use a moving average, and a value smaller than the actual value may be measured due to noise. However, the state in which Z>Cz2 is not satisfied even if the actual position of the fingertip FT is in the selection area is resolved in a time shorter than ProximityOffTH. Therefore, even if a small Z value is measured due to noise while the second pointer 112 is being displayed, the display of the second pointer 112 continues. On the other hand, even if the fingertip FT is actually moved to the proximity area, the display of the second pointer 112 continues for a while. Most users do not feel that the reaction is slow even if the time it takes for the display of the second pointer 112 to disappear is delayed. Therefore, the processing from S8B:NO to S9B can stabilize the operation without reducing the sensory reaction speed.

[0136] If the input support application 150 determines in step S8B that (current time-ProximityOffTime>ProximityOffTH) is true (S8B: YES), the input support application 150 sets the mode of the input support application 150 to the proximity mode (step S10B).

[0137] The input assistance application 150 causes the OS 140 to display an image and a message for the proximity mode (see FIG. 6B) (step S11B). As a result, as shown in FIG. 6B, the image and the message for the proximity mode are displayed on the display 110. After completing the process of step S11B, the input assistance application 150 advances the flow to step S3.

[0138] <s4b:no> Furthermore, if the input support application 150 determines in step S4B that the mode of the input support application 150 is not the selection mode (S4B: NO), the input support application 150 determines whether the Z value is greater than Cz2 (step S4C). That is, the input support application 150 determines whether the fingertip FT has entered the selection area again.

[0139] If the input assistance application 150 determines that the Z value is greater than Cz2 (S4C: YES), it sets ProximityOffTime to the current time (step S5C). As described above, if Z>Cz1 (S5 or S4A) is YES, or Z>Cz2 (S5B or S4C) is YES, ProximityOffTime is updated to the current time. Thus, the last time when Z was greater than Cz2 is stored as ProximityOffTime.

[0140] The input support application 150 sets the mode of the input support application 150 to the selection mode, and sets the current time in SelectionTime (step S6C).

[0141] The input assistance application 150 performs a process of setting the radius of the second pointer 112 to an initial value (step S7C). The process of setting the radius of the second pointer 112 to an initial value is a subroutine process, so the details will be described later, but the process is a process of setting the radius of the second pointer 112 to an initial value, not a radius according to the amount of variation of the XY coordinates. When the input assistance application 150 finishes the process of step S7C, the flow advances to step S21, and the second pointer 112 is displayed on the OS 140. Note that although the fingertip FT is actually in the proximity area, the Z value may be measured larger than the actual value due to noise, and the selection mode may be entered. However, since the selection mode is entered when the fingertip FT is somewhat close, most users do not recognize this as a malfunction. Conversely, when the fingertip FT is in a distant position (standby area), the selection mode is not entered even if a noise level that is normally present is entered.

[0142] <s4c:no> If the input support application 150 determines in step S4C that the Z value is not greater than Cz2 (S4C: NO), the input support application 150 determines whether the mode of the input support application 150 is the proximity mode (step S4D).

[0143] If the input support application 150 determines that the mode of the input support application 150 is the proximity mode (S4D: YES), the input support application 150 determines whether the Z value is greater than Cz3 (step S5D). If the Z value is greater than Cz2, the input support application 150 considers that the distance from the operation surface 105A to the fingertip FT is shorter than the third threshold value Z3.

[0144] If the input assistance application 150 determines that the Z value is greater than Cz3 (S5D: YES), the flow proceeds to step S11B. As a result, in step S11B, an image and a message for the proximity mode are displayed on the display 110 as shown in FIG. 6B.

[0145] <s5d:no> If the input support application 150 determines in step S5D that the Z value is not greater than Cz3 (S5D: NO), it sets the mode of the input support application 150 to the standby mode (step S6D). If the Z value is not greater than Cz3, the input support application 150 determines that the distance from the operation surface 105A to the fingertip FT is longer than the third threshold value Z3.

[0146] The input support application 150 causes the OS 140 to display an image and a message for the standby mode (step S7D). As a result, the image and the message for the standby mode are displayed on the display 110. After completing the process of step S7D, the input support application 150 advances the flow to step S3. Note that in the standby mode, the display may be turned off (S7D' in FIG. 7D).

[0147] <s4d:no> If the input support application 150 determines in step S4D that the mode of the input support application 150 is not the proximity mode (S4D: NO), the input support application 150 determines whether the Z value is greater than Cz3 (step S4E). The input support application 150 determines whether the fingertip FT has entered the proximity region again.

[0148] If the input support application 150 determines that the Z value is greater than Cz3 (S4E: YES), it sets the mode of the input support application 150 to the proximity mode (step S5E). The input support application 150 proceeds to step S11B and displays an image and a message for the proximity mode.

[0149] Furthermore, if the input support application 150 determines in step S4E that the Z value is not greater than Cz3 (S4E: NO), the flow proceeds to step S7D. In this case, since the input support application 150 is in the standby mode, an image and a message for the standby mode are displayed on the display 110 in step S7D.

[0150] This is the end of the series of main flows. The input assistance application 150 repeatedly executes the main flows. Next, each subroutine will be described.

[0151] <Figure 8A: Initial value setting> The input support application 150 sets the mode of the input support application 150 to the standby mode, and sets the flag null to YES (step S101).

[0152] The input assistance application 150 sets Nma to a predetermined integer (step S102). Nma is a calculated value of the number of multiple data used when calculating the moving average, and is calculated as Nma=int(MATime / Nmea). The specified integer is obtained as int(MATime / Nmea). int(MATime / Nmea) represents an integer obtained by rounding down the decimal point of MATime / Nmea. MATime is the time used for the moving average (the time from the start to the end), and Nmea is the measurement interval. That is, Nma=int(MATime / Nmea). The user of the non-contact input device 100 can set MATime for the input assistance application 150 through a setting screen displayed on the display 110.

[0153] The input assistance application 150 compares Nma with Ndev to determine which is larger (step S103). Ndev is the number of multiple data used when calculating an index of positional accuracy.

[0154] The input assistance application 150 sets the larger of Nma and Ndev as the number of arrays Num (steps S104 and S105). That is, the larger of the "number of data required to calculate the moving average" and the "number of data required to calculate the index of positional accuracy" is set as the number of arrays Num. Since the number of elements of the array is decided, the array may be declared at this stage. After completing the process of step S104, the input assistance application 150 ends the subroutine for setting initial values.

[0155] <Figure 8B: Coordinate calculation> The input assistance application 150 acquires the XY coordinates and the Z value from the input sensor circuit 125A (step S111).

[0156] The input assistance application 150 acquires the current time (present time) from the OS 140 (step S112). The value acquired in step S112 is used as the "present time" in the main loop described above.

[0157] The input support application 150 updates the array of X coordinates (step S113). Since step S113 is a subroutine process, the details will be described later. In step S113, the input support application 150 removes the oldest value from the array used for calculating the moving average of the X coordinates and the index of positional accuracy, and adds the latest value.

[0158] The input support application 150 updates the array of Y coordinates (step S114). Since step S113 is a subroutine process, the details will be described later. In step S114, the input support application 150 removes the oldest value from the array used for calculating the moving average of the Y coordinates and the index of positional accuracy, and adds the latest value.

[0159] The input assistance application 150 calculates a moving average from the array of X coordinates updated in step S113 (step S115).

[0160] The input assistance application 150 calculates the moving average of the array of the Y coordinates updated in step S114 (step S116).

[0161] The input assistance application 150 determines whether the Z value acquired in step S111 is greater than Cz3 (step S117). If the Z value is not greater than Cz3, the Z value is within the waiting area, and the reliability of the latest XY coordinate acquired in step S111 and the XY coordinates acquired in step S111 before that are low, and are not suitable coordinate values ​​for calculating the moving average of the XY coordinates, so the process of step S117 is provided.

[0162] If the input assistance application 150 determines that the Z value acquired in step S111 is greater than Cz3 (S117: YES), it sets the flag null to NO (step S118). Since the latest XY coordinates are coordinate values ​​suitable for calculating the moving average of the XY coordinates and the index of the positional accuracy, it sets the flag null to NO.

[0163] If the input assistance application 150 determines that the Z value acquired in step S111 is not greater than Cz3 (S117: NO), it sets the flag null to YES (step S119). Since the latest XY coordinates are not suitable for calculating the moving average of the XY coordinates, it sets the flag null to YES.

[0164] This completes the coordinate calculation subroutine.

[0165] <Figure 8C: Updating the X coordinate array> Fig. 8C shows details of the subroutine process for updating the X-coordinate array in step S113 in Fig. 8B. In the following, a plurality of X-coordinates acquired at a plurality of time points are distinguished as X(i). The times at which the plurality of X-coordinates X(i) were acquired are different from one another, and the plurality of X-coordinates are acquired continuously by the non-contact input device 100 at a predetermined sampling period. i = 1 to Num, and X(1) is the latest X-coordinate.

[0166] The input assistance application 150 determines whether the flag null is YES (step S121X) in order to determine whether the latest X coordinate is appropriate.

[0167] When the input assistance application 150 determines that the flag null is YES (S121X: YES), it performs a subroutine process of substituting the non-measured value into X(i) to create the X coordinate.

[0168] The input support application 150 selects Num-1 X coordinates (X(2) to X(Num)) where i ranges from 2 to Num one by one, and assigns a non-measured value to each of them (step S122X). A non-measured value is a value that is not obtained by measurement, and is a dummy value. Here, 0xFFFF is used as a non-measured value as an example. After completing the subroutine process including step S122X, the input support application 150 advances the flow to step S123X. Note that the non-measured value may be a value other than 0xFFFF, as long as it is a value that is not measured.

[0169] The input support application 150 assigns the latest X coordinate to X(1) (step S123X). In this manner, the input support application 150 updates Num X coordinates X(1) to X(Num).

[0170] Furthermore, if the input support application 150 determines that the flag null is NO (S121X: NO), it selects Num-1 X coordinates (X(1) to X(Num-1)) where i is 1 to Num-1 one by one, increments the i number by 1, and moves to the X coordinate (X(2) to X(Num)) (step S124X).

[0171] When the input support application 150 finishes the process of step S124X, the flow proceeds to step S123X. The input support application 150 updates the array of X coordinates X(1) to X(Num) by adding the X coordinate X(1) to the X coordinates (X(2) to X(Num)) acquired in step S124X.

[0172] <Figure 8D: Getting the latest Num Y coordinates> Fig. 8D shows details of the subroutine process for updating the Y coordinate array in step S114 of Fig. 8B. In the following, the multiple Y coordinates are distinguished as Y(i). The multiple Y coordinates Y(i) are acquired at different times, and are multiple Y coordinates acquired continuously by the non-contact input device 100 at a predetermined sampling period. The times when the multiple Y coordinates Y(i) are acquired are equal to the times when the multiple X coordinates X(i) are acquired, respectively. i = 1 to Num, and Y(1) is the latest Y coordinate.

[0173] The process of steps S121Y to S124Y shown in FIG. 8D is a process executed by the input assistance application 150 for the Y coordinate Y(i), and is a process in which the process of steps S121X to S124X for the X coordinate X(i) shown in FIG. 8C is replaced with a process for the Y coordinate Y(i). For this reason, the description of FIG. 8D is omitted here. Note that instead of the processes of S121X to S123X and S121Y to S123Y, a process of replacing the oldest value in the array with the latest value may be performed. In this case, it is not necessary to copy each element of the array. However, a subscript (index) specifying the element in which the oldest value is stored and a subscript (index) specifying the element in which the latest value is stored are required. In addition, it is also necessary to change the loop process when calculating the moving average and the amount of fluctuation, which will be described later.

[0174] <Figure 8E: Moving average of X coordinate> FIG. 8E shows details of the subroutine process for calculating the moving average of the X coordinate in step S115 of FIG. 8B.

[0175] The input assistance application 150 resets the integrated value Xacc of the X coordinate to 0 (step S131X).

[0176] The input assistance application 150 executes the processes of steps S132X, 133X, and 134X included in the subroutine for accumulating the X coordinate. In this subroutine, the i in X(i) is accumulated one by one from 1 to Nma.

[0177] The input assistance application 150 determines whether X(i) is 0xFFFF (step S132X), which means that there is no measured value of the Z value to be used in calculating the X coordinate.

[0178] When the input assistance application 150 determines that X(i) is 0xFFFF (S132X: YES), it updates the integrated value Xacc to Xacc+X(1) (step S133X). That is, when dummy data is stored in X(i), the latest X coordinate (X(1)) is used to calculate the integrated value. Therefore, the calculation of Xacc=Xacc+X(1) is performed.

[0179] Furthermore, if the input assistance application 150 determines in step S132X that X(i) is not 0xFFFF (S132X: NO), it updates the integrated value Xacc to Xacc+X(i) (step S134X). That is, it accumulates the previously measured X coordinates (X(i)) in order. For this reason, it calculates Xacc=Xacc+X(i).

[0180] When the input assistance application 150 repeatedly executes the processes of steps S132X, 133X, and 134X for i of X(i) from 1 to Nma, it ends the process of the subroutine for accumulating the X coordinate and advances the flow to step S135X.

[0181] The input assistance application 150 obtains the moving average of the X coordinate by dividing the integrated value Xacc by Nma (step S135X). That is, the moving average of the X coordinate Xave=Xacc / Nma.

[0182] <Figure 8F: Moving average of Y coordinate> FIG. 8F shows details of the subroutine process for calculating the moving average of the Y coordinate in step S116 of FIG. 8B.

[0183] The processing of steps S131Y to S135Y shown in Fig. 8F is processing executed by the input assistance application 150 for the Y coordinate Y(i), and is processing in which the processing of steps S131X to S135X for the X coordinate X(i) shown in Fig. 8E is replaced with processing for the Y coordinate Y(i). For this reason, the description of Fig. 8F will be omitted here.

[0184] <FIG. 9A: Processing for setting radius of second pointer 112 to initial value> FIG. 9A shows an example of the process of setting the radius of the second pointer 112 to an initial value in step S7C.

[0185] The input assistance application 150 sets the initial value of the radius Radius of the second pointer 112 to the maximum value Rmax (step S141A). That is, Radius=Rmax. The user of the non-contact input device 100 can set Rmax for the input assistance application 150 through a setting screen displayed on the display 110.

[0186] <FIG. 9B: Processing for setting radius of second pointer 112 to initial value> FIG. 9B shows another example of the process of setting the radius of the second pointer 112 to an initial value in step S7C.

[0187] The input assistance application 150 calculates the amount of variation D of the XY coordinates as an index of positional accuracy (step S141B). Step S141B is a subroutine process, and details will be described later with reference to FIG. 9E. As an example, the input assistance application 150 calculates the amount of variation D when the selection mode is entered. The input assistance application 150 sets the maximum value Rmax of the radius of the second pointer 112 to a value obtained by multiplying the amount of variation D of the XY coordinates by a constant (step S142B). That is, Rmax=D×constant. Note that the predetermined constant is a value determined in advance.

[0188] The input support application 150 sets the radius Radius of the second pointer 112 to the maximum value Rmax set in step S142B (step S143B). That is, Radius = Rmax = D x constant. Note that when performing the process of Fig. 9B, the input support application 150 does not perform the process of enabling the user to set the maximum value Rmax.

[0189] <FIG. 9C: Processing for setting the radius of the second pointer 112> FIG. 9C is a diagram showing an example of the second pointer radius setting process in steps S7B and S9B.

[0190] The input assistance application 150 calculates the amount of variation D of the XY coordinates as an index of positional accuracy (step S141C). Step S141C is a subroutine process, and the details will be described later with reference to FIG. 9E. As an example, the input assistance application 150 calculates the amount of variation D when the selection mode is entered.

[0191] The input assistance application 150 uses the amount of variation D calculated in step S141C to calculate the radius Radius of the second pointer 112 (step S142C). Radius Radius = Rmin + (Rmax - Rmin) x (D / Rmax). Note that Rmin is the minimum value of the radius of the second pointer 112, and is, for example, a predetermined value.

[0192] The input assistance application 150 determines whether the radius Radius calculated in step S142C is greater than the maximum value Rmax (step S143C).

[0193] If the input assistance application 150 determines that the radius Radius is greater than the maximum value Rmax (S143C: YES), it sets the radius Radius to the maximum value Rmax (step S144C). That is, Radius=Rmax.

[0194] Moreover, if the input assistance application 150 determines that the radius Radius is not greater than the maximum value Rmax (S143C: NO), it uses the calculated radius Radius as is.

[0195] As described above, as an example, the input assistance application 150 calculates the variation D when the selection mode is entered. When the second pointer 112 starts to be displayed in the selection mode, the size of the second pointer 112 gradually changes, making it easier to visually recognize the second pointer 112 of a size according to the positional accuracy.

[0196] For example, the input assistance application 150 may calculate the amount of variation D when the proximity mode is selected. By calculating the position accuracy before the second pointer 112 is displayed in the selection mode, the second pointer 112 can be displayed with a size according to the position accuracy from the time when the second pointer 112 starts to be displayed.

[0197] <Figure 9D: Calculation process of the amount of change D of the XY coordinates> Fig. 9D is a diagram showing an example of the calculation process of the amount of variation D of the XY coordinates. Fig. 9D shows an example of the calculation process of the amount of variation D in step S141C of Fig. 8C.

[0198] The input assistance application 150 resets the amount of variation DX of the X coordinate and the amount of variation DY of the Y coordinate (step S151A). That is, DX=0, DY=0.

[0199] The input support application 150 performs a subroutine process to calculate the accumulated amount of variation D. This subroutine includes steps S152A, S153A, and S154A, and is performed by setting i from 1 to Ndev for the X coordinate X(i) and the Y coordinate Y(i). That is, the input support application 150 calculates the amount of variation D using the latest Ndev number of X and Y coordinates.

[0200] The input assistance application 150 determines whether the X coordinate X(i) is 0xFFFF (step S152A).

[0201] If the input assistance application 150 determines that the X coordinate X(i) is 0xFFFF (S152A: YES), it adds a constant to the amount of variation DX of the X coordinate and the amount of variation DY of the Y coordinate (step S153A). That is, DX=DX+constant, and DY=DY+constant.

[0202] Furthermore, when the input assistance application 150 determines that the X coordinate X(i) is not 0xFFFF (S152A: NO), it adds the square of the difference between the moving average Xave of the X coordinate and the moving average Xave of the Y coordinate to the amount of fluctuation DX of the X coordinate and the amount of fluctuation DY of the Y coordinate (step S154A). That is, DX=DX+(X(i)-Xave). 2 , DY = DY + (Y(i) - Yave) 2 It becomes.

[0203] The input assistance application 150 includes steps S152A, S153A, and S154A, and when the steps are performed with i set from 1 to Ndev for the X coordinate X(i) and the Y coordinate Y(i), the flow proceeds to step S155A.

[0204] The input assistance application 150 uses DX and DY calculated in the subroutine process to calculate the amount of change D of the XY coordinates according to the following equation (2) (step S155A).

[0205]

number

[0206] This completes the determination of the amount of variation D. By determining DX and DY in step S154A and determining the amount of variation D in step S155A, the standard deviation of the XY coordinates can be determined as an index of positional accuracy.

[0207] The input assistance application 150 judges whether the fingertip FT is operated with the finger held up close to the operation surface 105A based on the amount of movement D (step S156A). If the amount of movement D is large, the fingertip FT is either far from the operation surface 105A, the finger is not extended, or the finger is laid down. If the amount of movement D is larger than the palm threshold (S156A: YES), a warning is displayed (step S157A). The warning is a message such as "Please hold your finger up and close to the operation surface to operate." In addition, the entire screen may be colored red along with the warning.

[0208] If the fingertip FT is suddenly brought closer to the operation surface 105A from the standby mode, the amount of variation D becomes large, and the position accuracy becomes poor. If the fingertip FT is kept close to the operation surface 105A, the dummy data substituted in step S122X is gradually replaced with the measurement data, and the amount of variation D becomes smaller. In this case, the position accuracy gradually improves. The amount of variation D may be obtained by the process shown in FIG. 9E.

[0209] <Figure 9E: Modified example of calculation process of the amount of change D in the XY coordinates> Fig. 9E is a diagram showing an example of the calculation process of the amount of variation D of the XY coordinates. Fig. 9D shows an example of the calculation process of the amount of variation D in step S141C of Fig. 8C.

[0210] The process shown in Fig. 9E is a process in which the content of step S155A in the process shown in Fig. 9D is changed. Therefore, the processes from steps S151B to S154B in Fig. 9E are the same as the processes from steps S151A to S154A in Fig. 9D. Therefore, the process of step S155B will be described here.

[0211] The input assistance application 150 uses DX and DY calculated in the subroutine process to calculate the amount of change D of the XY coordinates according to the following equation (3) (step S155A).

[0212] D=DX+DY (3)

[0213] In Fig. 9E, the fluctuation amount D is not divided by Ndev. For example, when Ndev is a constant, the sum (total) of DX and DY not divided by Ndev may be used as the fluctuation amount D. Also, instead of the fluctuation amount D of the XY coordinates, either DX or DY may be used as the fluctuation amount D. For example, either the fluctuation amount DX / Ndev of the X coordinate or the fluctuation amount DY / Ndev of the Y coordinate may be used.

[0214] <Figure 9F: Processing to determine whether or not to display the menu> Fig. 9F is a diagram showing an example of the process of determining whether or not the menu needs to be displayed, Fig. 9F shows the process of determining whether or not the menu needs to be displayed in step S23.

[0215] Here, Nmenu is the number of data corresponding to the time slot of the process of determining whether or not the menu needs to be displayed. Nmenu may be determined at the time of design, or may be set by the user from a setting screen. Also, Dmenu is the distance corresponding to the threshold of the movement amount of the X coordinate and the Y coordinate.

[0216] The input assistance application 150 performs a process of determining the X coordinate (determination X) while incrementing the i number of X(i) by 1 from 2 to Nmenu. The process of determining the X coordinate is a subroutine process.

[0217] The input assistance application 150 determines whether the absolute value of X(i)-X(1) is less than Dmenu (step S161), where X(1) is the latest X coordinate.

[0218] If the absolute values ​​of the differences between X(2) to X(Nmenu) and X(1) are all less than Dmenu (all S161: YES), the input assistance application 150 executes the subroutine process for the Y coordinate.

[0219] The input assistance application 150 performs a Y coordinate determination process (determination Y) while incrementing the i number of Y(i) by 1 from 2 to Nmenu. The Y coordinate determination process is a subroutine process.

[0220] The input assistance application 150 determines whether the absolute value of Y(i)-Y(1) is less than Dmenu (step S162), where Y(1) is the latest Y coordinate.

[0221] If the absolute values ​​of the differences between Y(2) to Y(Nmenu) and Y(1) are all less than Dmenu (all S162: YES), the input assistance application 150 advances the flow to step S163.

[0222] The input assistance application 150 determines whether or not the current time-SelectionTime>SelectionTH is satisfied (step S163). That is, if the time that has elapsed since the selection mode was entered exceeds the predetermined time, the flow proceeds to step S164.

[0223] The input assistance application 150 displays a menu (step S164). The menu is, for example, the function menu 112C (see FIG. 6E). If the finger is moving (S161: NO or S162: NO) or if the mode has just entered the selection mode (S163: NO), the function menu is not displayed.

[0224] The menu is displayed in the selection mode when the Nmenu number of X and Y coordinates are not moving (are stationary). That is, the input assistance application 150 causes the OS 140 to display the menu when the X and Y coordinates of the fingertip FT are stationary for a period of time during which the Nmenu number of X and Y coordinates are acquired in the selection mode. The period of time during which the Nmenu number of X and Y coordinates are acquired is an example of the third predetermined period of time.

[0225] <Settings screen> Fig. 10 is a diagram showing an example of a setting screen. The input assistance application 150 causes the OS 140 to display the setting screen shown in Fig. 10 on the display 110. The setting screen is an image representing a screen on which various settings can be input.

[0226] 10 includes five text boxes for setting the mode switching threshold, messages to be displayed on display 110 in standby mode and proximity mode, a moving average of the position of second pointer 112, a click event issuance time, a minimum and maximum value of the radius of second pointer 112, and text boxes for inputting a click re-output time. The threshold for the proximity mode corresponds to the Z value Cz3, the threshold for the selection mode corresponds to the Z value Cz2, and the threshold for the decision mode corresponds to the Z value Cz1.

[0227] 11 is a diagram showing an example of the configuration of folders. The folders are stored inside the memory 132 (see FIG. 2), and the input assistance application 150 can access the folders.

[0228] 11 shows the audio file (Click.wav) that can be selected on the setting screen and the background image files (Proximity.png, Waiting.png) for each mode. Note that, although the audio file and the background image file are shown in one folder here, the folders for the audio file and the background image file may be separate.

[0229] <Settings process flow chart> 12 is a flow chart showing an example of the setting process. When "Setting" is selected from the menu in the task tray, the input assistance application 150 starts a setting program and executes the process shown in FIG.

[0230] The input assistance application 150 causes the OS 140 to display a setting screen on the display 110 (step S201). The user can input various values ​​and the like into the setting screen shown in FIG.

[0231] The input assistance application 150 determines whether the apply button has been clicked (step S202).

[0232] When the input support application 150 determines that the Apply button has been clicked (S202: YES), it sets each value, etc. (step S203). Specifically, the input support application 150 inputs the value of the proximity mode setting box to ProximityTH, the value of the selection mode setting box to SelectionTH, the value of the decision mode setting box to DecisionTH, the value of the proximity mode OFF determination to ProximityOffTH, and the value of the selection mode OFF determination to SelectionOffTH. In addition, the input support application 150 inputs the value of the click re-output time to ReClickTH, the time of the second pointer movement average to MATime, the value of the click event issuance time to DecisionTimeTH, the minimum radius to Rmin, the maximum radius to Rmax, saves a message for the standby mode, and saves a message for the proximity mode.

[0233] If the input support application 150 determines in step S202 that the Apply button has not been clicked (S202: NO), the input support application 150 determines whether the OK button has been clicked (step S204). Note that the input support application 150 repeatedly executes the processes in and after step S202 until the OK button is clicked.

[0234] When the input support application 150 determines that the OK button has been clicked (S204: YES), it causes the OS 140 to close the setting screen (step S205). When the input support application 150 finishes the process of step S205, it ends the flow. Note that when the OK button has been clicked, the flow may end after each value, etc. has been set.

[0235] <Effects> The non-contact input device 100 includes an operation surface 105A, a detection unit (sensor electrodes 121X, 121Y, input sensor circuit 125A) that measures the two-dimensional position of an object operating on the operation surface 105A in a non-contact manner and the distance from the operation surface 105A to the object, a display 110 arranged on the back side of the operation surface 105A, and a control device 130 having an OS140 that uses a planar GUI and an input assistance application 150, the input assistance application 150 outputs to the OS140 a command to cause the OS140 to display a second pointer 112, which is different from a first pointer 111 displayed on the display 110, in an overlay on the display 110 according to the two-dimensional position and distance, and when it is determined that an operation by the object has been determined based on the two-dimensional position and distance, it outputs to the OS140 the two-dimensional position at the time the operation was determined and a click event.

[0236] Therefore, it is possible to provide a non-contact input device 100 that enables non-contact input with three-dimensional movements in a device equipped with an OS and application program that can only handle two-dimensional positions. Also, by having the application display a second pointer instead of a first pointer (mouse pointer) prepared by the OS, it is possible to display a second pointer that corresponds to a three-dimensional position even in an OS that displays a first pointer (mouse pointer) that only corresponds to a two-dimensional position.

[0237] The input assistance application 150 may output to the OS 140 a command to hide the first pointer 111. Without making any changes to the OS 140, a second pointer whose size, length of arc, etc. can be dynamically changed in response to an operational input can be displayed on the display 110 as a function not provided by the OS 140.

[0238] Further, a decision area (first area) where the distance from the operation surface 105A to the fingertip FT is shorter than the first threshold is a decision area where the input assistance application 150 executes the decision mode. If the measured capacitance is greater than the first threshold Cz1, the input assistance application 150 considers that the fingertip is in the decision area (first area). When the distance from the operation surface 105A of the fingertip FT (object) is located within the decision area (first area), the input assistance application 150 displays the second pointer 112 that moves according to the operation performed by the fingertip FT (object). When the distance from the operation surface 105A of the fingertip FT (object) remains within the decision area (first area), the input assistance application 150 displays, as the second pointer 112A, an arc that becomes longer with time and becomes a ring in a first predetermined time from rest. The input assistance application 150 may determine that the operation content has been decided when the time that the fingertip FT (target object) remains within the decision area (first area) from the operation surface 105A has elapsed for a first predetermined time, display a ring as the second pointer 112B, issue a click event to the OS 140, and output the two-dimensional position at the time when the operation content was decided to the OS 140. The user can intuitively understand that the operation content is decided when the fingertip FT is stopped for a certain period of time in the decision area.

[0239] Furthermore, when the distance from the operation surface 105A of the fingertip FT (target object) remains within the decision area (first area) after issuing a click event to the OS 140, the input assistance application 150 causes the OS 140 to display an arc that becomes longer with time and becomes a ring in a second predetermined time from the start of drawing the arc as the second pointer 112A, and when the second predetermined time has elapsed since issuing the click event, causes the OS 140 to display the ring as the second pointer 112B, and outputs the two-dimensional position and the click event to the OS 140. It is possible to provide a non-contact input device 100 that is easier to use because it enables continuous decision operations and allows non-contact input with three-dimensional movements.

[0240] Furthermore, the input assistance application 150 may cause the OS 140 to display an arc that gets longer with time as the fingertip FT (target object) from the operation surface 105A moves out of the decision area (first area) and then back into the decision area (first area), as the second pointer 112A. When successive decision operations are performed, the pointer can be displayed immediately.

[0241] Moreover, a selection area (second area) in which the distance from the operation surface 105A to the fingertip FT is equal to or greater than the first threshold and shorter than a second threshold that is longer than the first threshold is a selection area in which the input assistance application 150 executes the selection mode. When the object is located within the second area, the input assistance application 150 causes the OS 140 to display the second pointer 112 that moves according to the two-dimensional position of the object, and when the object stays within the second area for a third predetermined time and the two-dimensional position of the object stops, the input assistance application 150 causes the OS 140 to display a function menu in an overlay. When the fingertip FT (object) is located within the selection area (second area) from the operation surface 105A, the input assistance application 150 causes the OS 140 to display the second pointer 112A that moves according to an operation performed by the fingertip FT (object), and when the fingertip FT (object) stops within the selection area (second area) for a third predetermined time, the input assistance application 150 may cause the OS 140 to display the function menu in an overlay. As with the second pointer 112 (112A, 112B), the function menu can be displayed without making any changes to the OS 140.

[0242] Furthermore, when the distance of the fingertip FT (target object) from the operation surface 105A is within the selection region (second region), the input assistance application 150 may calculate the amount of change in the two-dimensional position calculated at multiple points in time as an index of the positional accuracy of the fingertip FT (target object), and may cause the OS 140 to display the second pointer 112 smaller as the positional accuracy of the fingertip FT (target object) improves. The second pointer 112 can be displayed smaller according to the positional accuracy by overlay without making any changes to the OS 140. Moreover, by displaying the second pointer 112 smaller according to the positional accuracy, the positional accuracy can be visually conveyed to the user.

[0243] Furthermore, the input assistance application 150 may transition from the decision mode to the selection mode when a fourth predetermined time has elapsed since the fingertip FT (target object) moves from the decision area (first area) to the selection area (second area) from the operation surface 105A. The decision mode can be stably switched to the selection mode.

[0244] Further, a proximity region (third region) in which the distance from the operation surface 105A to the fingertip FT is equal to or greater than the second threshold and shorter than a third threshold longer than the second threshold is a proximity region in which the input assistance application 150 executes the proximity mode. When the distance is within the third region, the input assistance application 150 outputs to the OS 140 a command to display an image or message for the proximity mode in an overlay on the display 110. When the two-dimensional position of the fingertip FT (target object) is within the proximity region (third region), the input assistance application 150 may output to the OS 140 a command to display an image or message for the proximity mode in an overlay on the display 110. Since the image or message for the proximity mode can be displayed in an overlay, the image or message for the proximity mode can be easily displayed to the user without making any changes to the OS 140. If an image showing an operation method is used as the image for the proximity mode, the operation method can be shown at an appropriate timing. Furthermore, by providing an operation screen on the input assistance application 150 side that allows easy replacement of images and messages, the administrator of the non-contact input device 100 can display images and messages on the display 110 that are in line with the application program.

[0245] The fourth area where the distance from the operation surface 105A to the fingertip FT is equal to or greater than the third threshold is a standby area where the input assistance application 150 does not calculate the two-dimensional position of the fingertip FT (object). When the object is not detected in the first area, the second area, and the third area, the input assistance application 150 outputs to the OS an instruction to display an image or message for standby mode in an overlay on the display 110. When the fingertip FT (object) is not detected in the decision area (first area), the selection area (second area), and the proximity area (third area), the input assistance application 150 may output to the OS 140 an instruction to display an image or message for standby mode in an overlay on the display 110. Images and messages for standby mode can be easily displayed to the user without making any changes to the OS 140. In addition, since images and messages can be easily replaced on the input assistance application 150 side, images and messages that conform to the application program can be displayed on the display 110.

[0246] Furthermore, the input assistance application 150 may calculate a moving average of two-dimensional positions acquired at multiple points in time as the two-dimensional position of the fingertip FT (object), and cause the OS 140 to display the second pointer 112 (112A, 112B) at the calculated two-dimensional coordinate position. By calculating a moving average of the X coordinate and the Y coordinate as the XY coordinate of the fingertip FT, the XY coordinate of the fingertip FT in which fluctuations due to external noise and hand movement are suppressed can be obtained, and the second pointer 112 (112A, 112B) can be displayed at highly accurate XY coordinates in which the influence of the amount of fluctuation is suppressed.

[0247] Furthermore, when the input assistance application 150 determines that the operation content by the fingertip FT (target object) has been confirmed, it may output to the OS 140 the two-dimensional position of the fingertip FT (target object) calculated as a moving average of the two-dimensional position when the operation content has been confirmed, and a click event. It is possible to output to the OS 140 that the confirmation operation has been performed based on the highly accurate XY coordinates calculated by the moving average, and it is possible to reflect the content of the confirmation operation with high accuracy. Furthermore, since the XY coordinates are not transmitted to the OS 140 until a click event is issued, it is possible to reduce the processing load on the OS 140.

[0248] The input assistance application 150 may also calculate the sum of the amount of variation of multiple two-dimensional positions as an index of the positional accuracy of the fingertip FT (target object) using multiple points in time where the number of pieces of data used for calculation is set by the user. The index of the positional accuracy according to the number of two-dimensional positions set by the user can be calculated. The user can adjust the level of the index of the positional accuracy.

[0249] In addition, the input assistance application 150 outputs to the OS 140 a command to display the second pointer 112 in an overlay on the display 110 at the size set by the user. The user can set the size of the second pointer 112 without making any changes to the OS 140. The user can set the size of the second pointer 112 via the input assistance application 150.

[0250] Furthermore, the input assistance application 150 may set the size of the second pointer to be proportional to the inverse of the Z value (a value proportional to the capacitance). In this case, too, it is possible to encourage the fingertip FT to approach the operation surface 105A. However, the size of the second pointer based on the Z value has no statistical meaning.

[0251] The above describes exemplary embodiments of the non-contact input device and input assistance application of the present disclosure. However, the present disclosure is not limited to the specifically disclosed embodiments, and various modifications and changes are possible without departing from the scope of the claims.

[0252] Regarding the above embodiment, the following supplementary notes are further disclosed. (Appendix 1) The operation surface and a detection unit that measures a two-dimensional position of an object performing a non-contact operation on the operation surface and a distance from the operation surface to the object; A display disposed on the rear side of the operation surface; A control device having an OS using a flat GUI and an input support application. Including, The input assistance application includes: outputting to the OS a command to display a second pointer, different from the first pointer displayed by the OS on the display, as an overlay on the display in accordance with the two-dimensional position and the distance; When it is determined that an operation by the object has been determined based on the two-dimensional position and the distance, the non-contact input device outputs the two-dimensional position at the time when the operation was determined and a click event to the OS. (Appendix 2) 2. The non-touch input device according to claim 1, wherein the input assistance application outputs a command to the OS to make the first pointer invisible. (Appendix 3) a first region in which the distance is shorter than a first threshold is a decision region in which the input assistance application executes a decision mode; The input assistance application includes: When the distance of the object is located within the first area, displaying the second pointer on the OS, the second pointer moving in response to an operation performed by the object; displaying, on the OS, as the second pointer, an arc that becomes longer over time when the distance of the object remains within the first region, the arc becoming a ring within a first predetermined time after the object enters the first region; 3. The non-contact input device according to claim 1, wherein when the time during which the distance of the object remains within the first area exceeds the first predetermined time, the device determines that the operation has been confirmed, displays the ring as the second pointer via the OS, issues a click event to the OS, and outputs the two-dimensional position at the time the operation was confirmed to the OS. (Appendix 4) The input assistance application includes: displaying, on the OS as the second pointer, an arc that becomes longer over time when the distance of the object continues to remain within the first region after a click event is issued to the OS and that becomes a ring within a second predetermined time from the start of drawing of the arc; A non-contact input device as described in Appendix 3, wherein when the second predetermined time has elapsed since the click event was issued, the ring is displayed on the OS as the second pointer, the click event is issued to the OS, and the two-dimensional position when the operation was confirmed and the click event are output to the OS. (Appendix 5) The non-touch input device according to claim 4, wherein the input assistance application causes the OS to display, as the second pointer, an arc that becomes longer over time when the distance of the object moves outside the first area and then returns to the first area. (Appendix 6) When the first predetermined time is set to 0, the input assistance application When the distance of the object is located within the first area, it is determined that the operation is confirmed; 6. The non-contact input device according to claim 3, wherein a click event is issued to the OS, and the two-dimensional position at the time when the operation is confirmed is output to the OS. (Appendix 7) The non-contact input device according to any one of claims 3 to 6, wherein the input assistance application causes the OS to display a circle smaller than the annular ring as the second pointer after the click event is output. (Appendix 8) a second region in which the distance is equal to or greater than the first threshold and shorter than a second threshold longer than the first threshold is a selection region in which the input assistance application executes a selection mode; The input assistance application includes: 8. The non-contact input device according to claim 3, wherein when a fourth predetermined time has elapsed since the object has moved from the first area to the second area, the non-contact input device transitions from a decision mode in which the click event is output to the selection mode. (Appendix 9) The input assistance application includes: calculating a sum of the amounts of change in the two-dimensional positions calculated at multiple time points when the object is located within the second region as an index of the positional accuracy of the object; The non-contact input device according to claim 8, wherein the OS is configured to display the second pointer smaller as the positional accuracy of the object improves. (Appendix 10) The input assistance application includes: When the distance of the object is located within the second region, displaying the second pointer on the OS, the second pointer moving according to the two-dimensional position of the object; 10. The non-contact input device according to claim 8, further comprising: when the distance of the object remains within the second region for a third predetermined time period and the two-dimensional position of the object becomes stationary, the OS is caused to display a function menu in an overlay. (Appendix 11) a third region in which the distance is equal to or greater than the second threshold and shorter than a third threshold longer than the second threshold is a proximity region in which the input assistance application executes a proximity mode; The non-contact input device according to any one of appendixes 8 to 10, wherein the input assistance application outputs, to the OS, a command to display an image or a message for a proximity mode as an overlay on the display when the distance is within the third area. (Appendix 12) a fourth region where the distance is equal to or greater than the third threshold is a standby region in which the input assistance application does not output the two-dimensional position of the object; The non-contact input device according to claim 11, wherein the input assistance application outputs an instruction to the OS to display an image or message for a standby mode as an overlay on the display when the object is not detected in the first area, the second area, or the third area. (Appendix 13) The input assistance application includes: calculating a moving average of the two-dimensional positions acquired at multiple time points as the two-dimensional position of the object; 13. The non-contact input device according to claim 9, further comprising: displaying, via the OS, the second pointer at the position of the calculated two-dimensional coordinates. (Appendix 14) The non-contact input device according to claim 13, wherein when the input assistance application determines that the operation by the object has been confirmed, the input assistance application outputs to the OS the two-dimensional position of the object calculated as a moving average of the two-dimensional position when the operation was confirmed and the click event. (Appendix 15) The non-contact input device according to claim 14, wherein the input assistance application calculates a sum of the amounts of variation of the multiple two-dimensional positions as an index of the positional accuracy of the object, using the multiple points in time at which a number of pieces of data to be used for calculation is set by a user. (Appendix 16) 16. The non-contact input device according to claim 1, wherein the input assistance application outputs to the OS a command to display the second pointer in an overlay on the display with a size set by a user. (Appendix 17) An input assistance application for operating an application program that runs on an OS that uses a planar GUI, using a detection unit that measures a two-dimensional position of an object that performs a non-contact operation input on an operation surface and a distance from the operation surface to the object, The input assistance application includes: outputting to the OS a command to display a second pointer, different from the first pointer displayed by the OS on the display, on the display in an overlay manner in accordance with the two-dimensional position and the distance; When it is determined that an operation by the object has been determined based on the two-dimensional position and the distance, the input assistance application outputs the two-dimensional position at the time when the operation has been determined and a click event to the OS. [Explanation of symbols]

[0253] 100 Non-contact input device 101 Case 105 Top Panel 105A Operation surface 110 Display 120 Electrostatic Sensor 121X, 121Y Sensor electrodes 125A Input Sensor Circuit 125B Image display circuit 130 Control device 131 Control Unit 132 Memory 140 OS 150 Input assistance applications 160 Application Programs

Claims

1. The operation surface and a detection unit that measures a two-dimensional position of an object performing a non-contact operation on the operation surface and a distance from the operation surface to the object; A display disposed on the rear side of the operation surface; A control device having an OS using a flat GUI and an input support application; Including, The input assistance application includes: outputting to the OS a command to display, on the display by the OS, a second pointer different from the first pointer displayed on the display in an overlay manner in accordance with the two-dimensional position and the distance; When it is determined that an operation by the object has been determined based on the two-dimensional position and the distance, the non-contact input device outputs the two-dimensional position at the time when the operation was determined and a click event to the OS.

2. The non-touch input device according to claim 1 , wherein the input assistance application outputs a command to the OS to make the first pointer invisible.

3. a first region in which the distance is shorter than a first threshold is a decision region in which the input assistance application outputs a click event; The input assistance application includes: When the distance of the object is located within the first area, displaying the second pointer on the OS, the second pointer moving in response to an operation performed by the object; displaying, on the OS, as the second pointer, an arc that becomes longer with time when the distance of the object remains within the first area, the arc becoming a ring within a first predetermined time after the object enters the first area; 2. The non-contact input device according to claim 1, wherein when the time that the distance of the object remains within the first area exceeds the first predetermined time, it is determined that the operation has been confirmed, the ring is displayed as the second pointer via the OS, a click event is issued to the OS, and the two-dimensional position at the time the operation was confirmed is output to the OS.

4. The input assistance application includes: displaying, on the OS as the second pointer, an arc that becomes longer over time when the distance of the object continues to remain within the first area after a click event is issued to the OS and that becomes a ring within a second predetermined time from the start of drawing of the arc; The non-contact input device according to claim 3 , wherein when the second predetermined time has elapsed since the click event was issued, the ring is displayed on the OS as the second pointer, and the two-dimensional position and the click event are output to the OS.

5. The non-contact input device according to claim 4 , wherein the input assistance application causes the OS to display, as the second pointer, an arc that becomes longer over time when the distance of the object moves outside the first area and then returns to the first area.

6. When the first predetermined time is set to 0, the input assistance application When the distance of the object is located within the first area, it is determined that the operation is confirmed; The non-contact input device according to claim 3 , wherein a click event is issued to the OS, and the two-dimensional position at the time when the operation is confirmed is output to the OS.

7. The non-contact input device according to claim 3 , wherein the input assistance application causes the OS to display a circle smaller than the annular ring as the second pointer after the click event is output.

8. a second region in which the distance is equal to or greater than the first threshold and shorter than a second threshold longer than the first threshold is a selection region in which the input assistance application executes a selection mode; The input assistance application includes: The non-contact input device according to claim 3 , wherein the input mode is changed from the decision mode in which the click event is output to the selection mode when a fourth predetermined time has elapsed since the object has moved from the first area to the second area.

9. The input assistance application includes: calculating a sum of the amounts of change in the two-dimensional positions calculated at multiple time points when the object is located within the second region as an index of positional accuracy of the object; The non-contact input device according to claim 8 , wherein the OS is configured to display the second pointer smaller as the positional accuracy of the object improves.

10. The input assistance application includes: When the distance of the object is located within the second area, the second pointer is displayed on the OS, the second pointer being moved according to the two-dimensional position of the object; The non-touch input device according to claim 8 , further comprising: when the distance of the object remains within the second region for a third predetermined time and the two-dimensional position of the object becomes stationary, the OS is caused to display a function menu in an overlay.

11. a third region in which the distance is equal to or greater than the second threshold and shorter than a third threshold longer than the second threshold is a proximity region in which the input assistance application executes a proximity mode; The non-touch input device according to claim 8 , wherein the input assistance application outputs to the OS a command to display an image or a message for a proximity mode as an overlay on the display when the distance is within the third region.

12. a fourth region where the distance is equal to or greater than the third threshold is a standby region in which the input assistance application does not output the two-dimensional position of the object; The non-touch input device according to claim 11 , wherein the input assistance application outputs an instruction to the OS to display an image or message for a standby mode as an overlay on the display when the object is not detected in the first area, the second area, and the third area.

13. The input assistance application includes: calculating a moving average of the two-dimensional positions acquired at multiple time points as the two-dimensional position of the object; The non-contact input device according to claim 9 , wherein the second pointer is displayed at the calculated two-dimensional coordinate position via the OS.

14. The non-contact input device according to claim 13, wherein when the input assistance application determines that the operation by the object has been confirmed, the input assistance application outputs to the OS the two-dimensional position of the object calculated as a moving average of the two-dimensional position when the operation was confirmed and the click event.

15. The non-contact input device according to claim 14 , wherein the input assistance application calculates a sum of the amounts of change in the two-dimensional positions as an index of the positional accuracy of the object, using the multiple points in time at which a number of pieces of data to be used for calculation is set by a user.

16. The non-touch input device according to claim 1 , wherein the input assistance application outputs to the OS a command to display the second pointer in an overlay on the display with a size set by a user.

17. An input assistance application for operating an application program that runs on an OS that uses a flat GUI, using a detection unit that measures a two-dimensional position of an object that performs a non-contact operation input on an operation surface and a distance from the operation surface to the object, The input assistance application includes: outputting to the OS a command to display, on the display, a second pointer different from the first pointer displayed by the OS in an overlay manner in accordance with the two-dimensional position and the distance; When it is determined that an operation by the object has been determined based on the two-dimensional position and the distance, the input assistance application outputs the two-dimensional position at the time when the operation has been determined and a click event to the OS.

Citation Information

Patent Citations

  • Information display device

    JP2010244422A

  • Imager device with opaque layer blocking light from the peripheral circuitry and its manufacturing method

    WO2007064771A2