Non-contact input device

Abstract

This invention provides a non-contact input device capable of calculating positional accuracy in response to external noise and hand movements. [Solution] The non-contact input device of the embodiment of the present disclosure includes a plurality of sensor electrodes arranged two-dimensionally along a first axis and a second axis on the back side of the operating surface, an input sensor circuit that measures the two-dimensional position of an object and the distance from the operating surface to the object from the capacitance between each of the plurality of sensor electrodes and an object to which non-contact operation input is performed on the operating surface, a display arranged in conjunction with the plurality of sensor electrodes, and a control device, and calculates the sum of the fluctuation amounts of a plurality of two-dimensional positions calculated at a plurality of time points as an indicator of the positional accuracy of the object.

Description

Technical Field

[0001] The present disclosure relates to a non-contact input device.

Background Art

[0002] Conventionally, in a display input device including a display device that displays an operation target image on a display surface and a touch panel that detects a contact position of an operation indicator (object) on the display surface, distance detecting means for detecting a distance in a direction perpendicular to the touch panel surface between the object and the touch panel and two-dimensional coordinates of the touch panel surface at a position (detection position) where the distance is detected, and pointer generation means for generating pointer data displayed at the two-dimensional coordinates of the detection position on the display surface. The pointer generation means generates the pointer data so that the pointer becomes smaller as the distance becomes shorter (becomes larger as the distance becomes longer) (see, for example, Patent Document 1, Patent Document 2, and Patent Document 3). Incidentally, it is known to change the color or shape of the pointer according to the distance (see, for example, Patent Document 4). Also, distance detecting means for detecting two-dimensional coordinates and a distance using a capacitance sensor is known (for example, Patent Document 5).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, the reason why the pointer size increases as the distance between the object and the touch panel surface increases (and decreases as the distance decreases) is based on the premise that the accuracy of object detection is worse at longer distances and higher at shorter distances.

[0005] Therefore, the objective is to provide a non-contact input device that can provide feedback on positional accuracy in response to external noise and hand movements. [Means for solving the problem]

[0006] The non-contact input device of the embodiment of the present disclosure includes a plurality of sensor electrodes arranged two-dimensionally along a first axis and a second axis on the back side of the operating surface, an input sensor circuit that measures the two-dimensional position of an object and the distance from the operating surface to the object from the capacitance between each of the plurality of sensor electrodes and an object to which non-contact operation input is performed with respect to the operating surface, a display arranged in conjunction with the plurality of sensor electrodes, and a control device, which calculates the sum of the fluctuation amounts of a plurality of two-dimensional positions calculated at a plurality of time points as an indicator of the positional accuracy of the object. [Effects of the Invention]

[0007] This device provides a non-contact input device capable of calculating positional accuracy in response to external noise and hand movements. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the configuration of a contactless input device according to an embodiment. [Figure 2] This figure shows an example of the configuration of an electrostatic sensor and control device, etc., of a non-contact input device according to an embodiment. [Figure 3] This figure shows an example of four regions corresponding to the distance in the Z direction from the operating surface of the non-contact input device of the embodiment. [Figure 4]It is a diagram for explaining the OS, input support application, and predetermined application program of the control unit of the contactless input device of the embodiment. [Figure 5A] It is a diagram for explaining an index of the position accuracy of an object in the contactless input device of the embodiment. [Figure 5B] It is a diagram for explaining an index of the position accuracy of an object in the contactless input device of the embodiment. [Figure 5C] It is a diagram for explaining an index of the position accuracy of an object in the contactless input device of the embodiment. [Figure 6A] It is a diagram showing an example of the display on the display of the contactless input device of the embodiment. [Figure 6B] It is a diagram showing an example of the display on the display of the contactless input device of the embodiment. [Figure 6C] It is a diagram showing an example of the display on the display of the contactless input device of the embodiment. [Figure 6D] It is a diagram showing an example of the display on the display of the contactless input device of the embodiment. [Figure 6E] It is a diagram showing an example of the display on the display of the contactless input device of the embodiment. [Figure 6F] It is a diagram showing an example of the display on the display of the contactless input device of the embodiment.​​​​​​​​​​​​​​​​​It is a diagram showing an example of the processing executed by the input support application of the contactless input device of the embodiment. [Figure 8C] It is a diagram showing an example of the processing executed by the input support application of the contactless input device of the embodiment. [Figure 8D] It is a diagram showing an example of the processing executed by the input support application of the contactless input device of the embodiment. [Figure 8E] It is a diagram showing an example of the processing executed by the input support application of the contactless input device of the embodiment. [Figure 8F] It is a diagram showing an example of the processing executed by the input support application of the contactless input device of the embodiment. [Figure 9A] It is a diagram showing an example of the processing executed by the input support application of the contactless input device of the embodiment. [Figure 9B] It is a diagram showing an example of the processing executed by the input support application of the contactless input device of the embodiment. [Figure 9C] It is a diagram showing an example of the processing executed by the input support application of the contactless input device of the embodiment. [Figure 9D] It is a diagram showing an example of the processing executed by the input support application of the contactless input device of the embodiment. [Figure 9E] It is a diagram showing an example of the processing executed by the input support application of the contactless input device of the embodiment. [Figure 9F] It is a diagram showing an example of the processing executed by the input support application of the contactless input device of the embodiment. [Figure 10] It is a diagram showing an example of the setting screen. [Figure 11] It is a diagram showing an example of the folder configuration. [Figure 12] It is a flowchart showing an example of the setting process.

Mode for Carrying Out the Invention

[0009] The following describes embodiments applying the contactless input device and input support application of this disclosure.

[0010] <Embodiment> Figure 1 is a diagram showing an example of the configuration of the contactless input device 100 according to the embodiment. Figure 1 shows the state in which the display 110 is displaying an input image.

[0011] Figure 2 shows an example of the configuration of the electrostatic sensor 120 and control device 130 of the non-contact input device 100. Figure 3 shows four regions corresponding to the distance in the Z direction from the operating surface 105A of the non-contact input device 100.

[0012] The following explains the XYZ coordinate system. The X-axis is the first axis, the Y-axis is the second axis, and the Z-axis is the third axis. The directions parallel to the X-axis (X direction), the directions parallel to the Y-axis (Y direction), and the directions parallel to the Z-axis (Z direction) are mutually orthogonal. Furthermore, in the following explanation, the -Z direction is described as the direction approaching the electrostatic sensor 120, and the +Z direction is described as the direction away from the electrostatic sensor 120. Plane view refers to viewing from the XY plane. Also, in the following explanation, the length, width, thickness, etc. of each part may be exaggerated to make the configuration easier to understand.

[0013] The contactless input device 100 may be, for example, a tablet-type input device or an input unit of an ATM (Automatic Teller Machine) that is placed in a store or facility and used by an unspecified number of users. It may also be an input unit of a cooking appliance that needs to be kept clean.

[0014] <Overall configuration of the non-contact input device 100> The non-contact input device 100 includes a housing 101, a top panel 105, a display 110, an electrostatic sensor 120, an input sensor circuit 125A, an image display circuit 125B, and a control device 130. The input sensor circuit 125A calculates XYZ coordinates from the measurement results of the electrostatic sensor 120. In Figure 1, the control device 130 (see Figure 2) is omitted, but as an example, the control device 130 is located inside the housing 101, below the display 110 and the electrostatic sensor 120. The non-contact input device 100 includes the electrostatic sensor 120 and the control device 130 shown in Figure 2.

[0015] <Chassis 101 and top panel 105> The housing 101 is a case made of resin or metal, etc., that houses the display 110, the electrostatic sensor 120, and the control device 130. The display 110 is positioned below the transparent electrostatic sensor 120 and is visible through the operating surface 105A, which is the upper surface of the transparent top panel 105 provided in an opening at the top of the housing 101. The electrostatic sensor 120 may be retrofitted to an existing display 110. The display and the control device 130 may be integrated, as in a tablet computer. Alternatively, the display and the control device 130 may be separate, as in a desktop computer.

[0016] <Four areas and operating methods of the non-contact input device 100> The non-contact input device 100 can be operated in both states: when the user's hand or other indicator is not in contact with the operating surface 105A, and when the user's hand or other indicator is in contact with the operating surface 105A. This will be explained in conjunction with the modes executed by the input support application of the non-contact input device 100. There are four modes for the input support application: confirmation mode, selection mode, proximity mode, and standby mode.

[0017] The non-contact input device 100 is an input device operated by the user through a pointing operation. A pointing operation is an operation performed by holding a finger approximately perpendicular to the operating surface 105A. Multiple fingers may be used for the pointing operation, but one finger is preferred.

[0018] When performing such a pointing operation, if the finger is not approximately perpendicular to the operating surface 105A, the entire palm will be closer to the operating surface 105A, making it difficult to measure the position of the fingertip FT.

[0019] In the following, unless otherwise specified, we will describe the case where the user performs a pointing operation using their fingertip FT as a pointing object. Furthermore, in the following, performing an operation (proximity operation, selection operation, decision operation, or touch operation) with the fingertip FT will be simply referred to as "operating with the fingertip FT (proximity operation, selection operation, decision operation, or touch operation)."

[0020] There are three types of operation methods for the non-contact input device 100: proximity operation, selection operation, and confirmation operation.

[0021] The non-contact input device 100 uses the four regions shown in Figure 3 to distinguish between four operating methods. The four regions, in order from the operating surface 105A side, are the determination region, the selection region, the proximity region, and the standby region.

[0022] <Decision area, selection area, proximity area, and waiting area> The decision region is the first region where the distance in the Z direction from the operating surface 105A is shorter than Z1 (e.g., 2 cm), and it is the region where the input support application sets the input support application mode to decision mode. Z1 is an example of the first threshold. When the input support application determines that the fingertip FT is located within the decision region, a decision operation is performed. Note that a distance of 0 cm (contact) in the Z direction from the operating surface 105A is also included in the decision region. Note that the relationship between the distance in the Z direction and capacitance is affected by individual differences and the installation environment. However, if the operator and the usage environment are the same, capacitance is inversely proportional to the distance in the Z direction. Therefore, if it is sufficient to measure the relative distance to the same operator, capacitance can be considered as distance. Details of the decision region and decision operation will be described later.

[0023] The input support application uses a value (Z value) proportional to the capacitance between the fingertip FT and the electrostatic sensor 120 instead of the distance in the Z direction. The input sensor circuit 125A calculates the position (XY coordinates) of the fingertip FT facing the operating surface 105A and a value (Z value) proportional to the capacitance from the operating surface 105A to the fingertip FT, based on the capacitance at multiple intersections of multiple sensor electrodes 121X and multiple sensor electrodes 121Y.

[0024] The determination region is the region where the Z value is greater than Cz1 (e.g., 60). Cz1 is a value proportional to the capacitance corresponding to the distance Z1. Cz1 is an example of a first threshold. In this embodiment, the case where the fingertip FT touches the operating surface 105A is also included in the determination region. However, it is also possible to determine from the Z value whether the fingertip FT has touched the operating surface 105A. For example, it is possible to design the system so that the Z value becomes 100 when the fingertip FT lightly touches the operating surface 105A. If the Z value is greater than 100, it may be determined that the fingertip FT is in the contact region and the system may operate in contact mode.

[0025] The selection region is a second region where the distance from the operating surface 105A is greater than or equal to Z1, and less than Z2 (e.g., 4 cm), which is longer than Z1. This region is where the input support application sets its mode to selection mode. Z2 is an example of a second threshold. When the input support application determines that the fingertip FT is located within the selection region, a selection operation is performed. Details of the selection operation will be described later.

[0026] The selection region is the region where the Z value is greater than Cz2 (e.g., 40) and less than or equal to Cz1. Cz2 is a value less than Cz1. Cz2 and Cz1 are values ​​proportional to the capacitance corresponding to distances Z2 and Z1, respectively. Capacitance Cz2 is an example of a second threshold.

[0027] The proximity region is a third region where the distance from the operating surface 105A is Z2 or greater, and shorter than Z3 (e.g., 7 cm), which is longer than Z2. This is the region where the input support application sets the application mode to proximity mode. Z3 is an example of a third threshold. When the input support application determines that the fingertip FT is located within the proximity region, a proximity operation is performed. The proximity region is the region furthest from the operating surface 105A among the regions where the XY coordinates of the object can be calculated. Details of the proximity operation will be described later.

[0028] The near-field region is the region where the Z value is greater than Cz3 (e.g., 10) and less than or equal to Cz2. Cz3 is a value less than Cz2. Cz3 and Cz2 are values ​​proportional to the capacitance corresponding to distances Z3 and Z2, respectively. Cz3 is an example of a third threshold.

[0029] The standby area is the fourth area where the distance from the operating surface 105A is longer than Z3, and it is the area where the input support application sets the input support application mode to standby mode. The standby area is the area where the Z value is Cz3 or less. Note that the capacitance value measured when there is no fingertip FT around the operating surface 105A is equal to the reference value. Calibration is performed so that the Z value when there is no fingertip FT around the operating surface 105A is 0. The input support API may turn off the power to the display in standby mode.

[0030] <Proximity operations, selection operations, decision operations, and contact operations> Proximity operation is an operation in which the fingertip FT is brought close to the operating surface 105A of the non-contact input device 100 without touching it, and is an operation to switch the display 110 from standby mode to proximity mode.

[0031] A selection operation is an operation in which, after performing a proximity operation, the fingertip FT is brought even closer to the operating surface 105A of the non-contact input device 100 without touching the operating surface 105A, and a GUI button displayed on the display 110 is selected.

[0032] A decision operation is an operation in which, after performing a selection operation, the fingertip FT is brought closer to the operating surface 105A of the non-contact input device 100 without touching it, thereby issuing a click event. When a click event is issued on a GUI button, the OS 140 and the application program 160 decide on the operation input. A decision operation is a non-contact operation input, meaning that the non-contact input device 100 is operated without touching the operating surface 105A with the fingertip FT. Operation inputs performed by non-contact selection and decision operations may also be called hover input or touchless input. Note that even if the fingertip FT touches the operating surface 105A of the non-contact input device 100, it may be treated as a decision mode. Alternatively, if the fingertip FT touches the operating surface 105A of the non-contact input device 100, the input may be confirmed immediately. Alternatively, if the fingertip FT touches the operating surface 105A of the non-contact input device 100, a warning screen may be displayed.

[0033] <Display 110> The display 110 is an LCD display or an OLED (Electroluminescence) display, etc. The display 110 is a display for realizing a GUI (Graphical User Interface). The display 110 displays an image provided by the OS's graphical shell (desktop screen, etc.: not shown), a second pointer 112 that is overlaid by the input support application 150, and an image 115 of the application software. The input support application 150 overlays the image by setting the Z-order of the second pointer 112 to the highest level (closest to the viewpoint). The image 115 of the application software includes an image of a GUI button 110A. The GUI button 110A is an operation part, and for example, it is shaped like a push button. The application software is designed to be operated with a two-dimensional position input device such as a mouse.

[0034] Figure 1 shows an example of a restaurant ordering terminal screen. The ordering screen shows a total of 17 GUI buttons 110A: 8 GUI buttons 110A for menus and 9 GUI buttons 110A in the form of a numeric keypad. The 17 GUI buttons 110A are arranged in 3 or 4 rows in the Y direction and 5 rows in the X direction. The rows extend in the X direction, and the Y direction extends in the column direction. Note that the GUI buttons 110A are not limited to restaurant ordering terminal screens; they can be used in any touch panel operation screen of other applications.

[0035] <Electrostatic sensor 120, input sensor circuit 125A, image display circuit 125B> The electrostatic sensor 120 is placed on top of the display 110 and, as shown in Figure 2, has a plurality of sensor electrodes 121X extending in the X direction and a plurality of sensor electrodes 121Y extending in the Y direction. An input sensor circuit 125A is integrally provided with the electrostatic sensor 120. An image display circuit 125B is integrally provided with the display 110. The input sensor circuit 125A is connected between the wiring 122X, 122Y and the control device 130. The image display circuit 125B is connected between the display 110 and the control device 130.

[0036] The sensor electrodes 121X and 121Y are connected to the control device 130 via wiring 122X and 122Y and the input sensor circuit 125A. Such an electrostatic sensor 120 can be made by forming a transparent conductive film such as ITO (Indium Tin Oxide) on the surface of transparent glass and patterning it onto the sensor electrodes 121X and 121Y and the wiring 122X and 122Y. The capacitance detected by the electrostatic sensor 120 is input to the control device 130.

[0037] Figure 2 shows multiple sensor electrodes 121X and multiple sensor electrodes 121Y. It is preferable that the spacing between sensor electrodes 121X and sensor electrodes 121Y is narrower than the spacing between GUI buttons 110A. In other words, it is preferable to use electrostatic sensors 120 that correspond to the spacing between GUI buttons 110A.

[0038] The input sensor circuit 125A is mounted on the wiring board. The input sensor circuit 125A is located between the wiring 122X and 122Y and the control device 130, and performs analog-to-digital (AD) conversion of the capacitance of the electrostatic sensor 120 obtained by sequentially selecting multiple wirings 122X and multiple wirings 122Y. From the capacitance values ​​of each wiring, the input sensor circuit 125A calculates the XY coordinates of the fingertip FT and the Z value which is proportional to the capacitance value between the operating surface 105A and the fingertip FT. The input sensor circuit 125A is capable of XY output in the same format as a contact-operated digitizer, as well as XYZ output in a unique format. The input support application 150 sends a command to the input sensor circuit 125A via the coordinate device driver to stop the digitizer-format XY output. The input support application 150 performs processing based on the XYZ output. The image display circuit 125B is located between the display 110 and the control device 130, and displays an image on the display 110 according to the image data sent from the control device 130.

[0039] <Control device 130> The control device 130 comprises a control unit 131 and a memory 132. The control device 130 is implemented by a computer including a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), input / output interface, and internal bus. The control unit 131 represents the functions of the program executed by the control device 130 as functional blocks. The memory 132 functionally represents the memory of the control device 130.

[0040] The control unit 131 controls the operation of the contactless input device 100. The control unit 131 receives the XY coordinates and Z value input from the input sensor circuit 125A. The control unit 131 controls the display of images on the display 110 via the image display circuit 125B. The control unit 131 outputs commands corresponding to the operation content determined by the fingertip FT decision operation. For example, if the contactless input device 100 is an ordering terminal in a restaurant, it transmits the name and quantity of the dishes ordered by the customer to the order management terminal in the kitchen.

[0041] The control unit 131 includes an OS using a planar GUI, a coordinate device driver, a display device driver, an input support application, and a predetermined application program. To illustrate the OS, input support application, and predetermined application program, Figure 4 is used. Figure 4 is a diagram illustrating the OS, input support application, and predetermined application program of the control unit 131.

[0042] Figure 4 shows the OS 140, input support application 150, and application program 160 of the control unit 131. The OS 140 is an operating system (OS) that uses a flat GUI, such as Windows® or Android®. OSs that use a flat GUI do not provide a UI (User Interface) suitable for 3D input. The application program 160 is an application program for a restaurant ordering terminal, etc.

[0043] OS140 has a coordinate device driver 141 and a display device driver 142. The coordinate device driver 141 passes the XY coordinates and Z value input from the electrostatic sensor 120 to the input support application 150. When OS140 receives XY coordinates and a click event from the input support application 150, it causes the application program 160 to execute a command corresponding to the button on the input XY coordinates. It also passes the data of the image to be displayed, the XY position of the image to be displayed, and the Z order (order of distance from the viewpoint) of the image to be displayed to the display device driver 142. The Z order of the second pointer 112 is set to the highest by the input support application 150.

[0044] The display device driver 142 controls the display of the second pointer 112 at the XY coordinates input from the input support application 150, and controls the display of an image according to the instructions input from the application program 160 when a decision operation is performed. The OS treats the second pointer 112 as a simple image.

[0045] The input support application 150 is an application that provides input support to users using the contactless input device 100. The input support application 150 may also perform a process to calculate an index of the positional accuracy of the fingertip FT. Based on the XY coordinates of the fingertip FT and the index of positional accuracy, the input support application 150 performs a process to display an image of a second pointer 112, which is different from the first pointer 111 that the OS 140 displays on the display 110. In addition, when a decision operation is performed, the input support application 150 outputs the XY coordinates at the time the decision operation was performed as the mouse pointer position to the OS 140 and then issues a click event. The OS 140 notifies the application program 160 that a GUI button corresponding to the mouse pointer position at the time the click event was issued has been pressed. Other processes performed by the input support application 150 and details of each process will be described later.

[0046] Application program 160 is designed to be operated using a standard mouse (which outputs two-dimensional coordinates) or a touch panel. When application program 160 receives notification from OS 140 that a click event has been issued on a GUI button, it performs processing corresponding to the clicked GUI button.

[0047] The display 110, control unit 130, OS 140, display device driver 142, and application program 160 can be existing devices that do not support 3D input (non-contact input). By fixing a non-contact electrostatic sensor 120 to the existing display 110 and installing a coordinate device driver 141 and an input support application 150 on the control unit 130, the existing device becomes a non-contact input device.

[0048] <Calculation of XY coordinates and Z value by input sensor circuit 125A> The input sensor circuit 125A scans multiple sensor electrodes 121X one row at a time and multiple sensor electrodes 121Y one column at a time, and converts the capacitance at multiple intersections of the multiple sensor electrodes 121X and multiple sensor electrodes 121Y into digital values. The control unit 131 counts the change in the output of the digital capacitance and calculates the difference value ΔAD at each intersection. The difference value ΔAD is the count value of the change in the output of the input sensor circuit 125A relative to a reference value. The reference value is a value proportional to the capacitance at each intersection of sensor electrodes 121X and 121Y when there are no objects such as a fingertip FT around the sensor electrodes 121X and 121Y. From the difference value at each intersection, the input sensor circuit 125A calculates the XY coordinates on the display facing the fingertip FT and selects the largest ΔAD among the difference values ​​ΔAD at each intersection as the Z value. The Z value is a value proportional to the capacitance between the fingertip FT and the operating surface. The XY coordinates and Z value may be calculated by the control unit 131.

[0049] Furthermore, it is possible to increase the resolution by interpolation, using the spacing between sensor electrodes 121X and sensor electrodes 121Y. In this case, the spacing between sensor electrodes 121X and sensor electrodes 121Y may be wider than the spacing between GUI buttons 110A. The coordinate device driver 141 may calculate the XY coordinates and Z value. When the coordinate device driver 141 calculates the XY coordinates and Z value, the hardware that performs the calculation is the control unit 131.

[0050] <Indicators of positional accuracy of objects> The input support application 150 determines an index of the positional accuracy of an object when displaying the second pointer 112 on the display 110. The object is an object whose XY coordinates and distance in the Z direction from the operating surface 105A can be determined based on the capacitance between it and the electrostatic sensor 120, and is a part of the user's body, such as their hand or fingertips, when using the non-contact input device 100. Here, we will explain the index of the positional accuracy of the object.

[0051] Figures 5A and 5B illustrate indicators of the positional accuracy of an object. The non-contact input device 100 uses the amount of variation in the XY coordinates (two-dimensional position) of the fingertip FT as an indicator of the positional accuracy of the object.

[0052] Figure 5A shows an example of the distribution of capacitance in the X direction measured at times t1 and t2. Assume that the X coordinate of the fingertip FT is calculated as X1 at time t1, and the X coordinate of the fingertip FT is calculated as X2 at time t2.

[0053] Even without moving the fingertip FT, the X-coordinate fluctuates due to noise. There is a difference between the X-coordinate X1 at time t1 and the X-coordinate X2 at time t2, and this difference in coordinates appears as the amount of coordinate fluctuation. The same applies to the Y-coordinate.

[0054] The non-contact input device 100 uses the amount of coordinate variation measured within a certain period of time as an indicator of positional accuracy. More specifically, it uses the standard deviation of the measured coordinates as an indicator of positional accuracy. For example, if the second pointer 112 is a circle with a radius of 1σ of the standard deviation, the probability of a finger being within the second pointer 112 is approximately 68%, and if the second pointer 112 is a circle with a radius of 2σ, the probability of a finger being within the second pointer 112 is approximately 95%. In this way, by representing positional accuracy with the size of the pointer, it is possible to provide intuitive feedback on positional accuracy to the user.

[0055] Figure 5B shows an example of the capacitance distribution in the X direction when a fingertip FT is held upright against the operating surface 105A and gradually brought closer to the operating surface 105A. In Figure 5B, the left fingertip FT is furthest from the operating surface 105A, and the right fingertip FT is closest to the operating surface 105A. The double arrows in Figure 5B indicate the range in which capacitance is detected. Note that while Figure 5B shows the capacitance distribution in the X direction, the capacitance distribution in the Y direction is similar due to symmetry, as explained in Figure 5C.

[0056] When the fingertip FT is far from the operating surface 105A (left side), the electrostatic sensor 120 is affected by the fingertip FT over a wide area, resulting in a flat capacitance distribution. If the noise magnitude is equal to the distance between the two dotted lines, the range in which the fingertip FT can be detected as the X coordinate is the range in which the capacitance is greater than the lower dotted line. In other words, the range in the X direction in which the fingertip FT can be detected is the wide range illustrated by the left and right arrows.

[0057] Furthermore, when the fingertip FT is close to the operating surface 105A (on the right side), the electrostatic sensor 120 is affected by the fingertip FT over a narrow range, resulting in a steep capacitance distribution. If the noise magnitude is equal to the distance between the two dotted lines, the range in which the fingertip FT can be detected as the X coordinate is the range in which the capacitance is greater than the lower dotted line. In other words, the range in the X direction in which the fingertip FT can be detected is the narrow range illustrated by the left and right arrows.

[0058] Furthermore, as shown in the center of Figure 5B, when the distance from the fingertip FT to the operating surface 105A is intermediate, the capacitance distribution in the X direction becomes an intermediate distribution between the gentle distribution on the left and the steep distribution on the right.

[0059] Thus, the range over which the measured XY coordinates fluctuate due to noise is larger when the fingertip FT is farther from the operating surface 105A (left side) than when the fingertip FT is closer to the operating surface 105A (right side). The larger the area over which capacitance is detected, the greater the amount of coordinate fluctuation.

[0060] Figure 5C shows an example of the capacitance distribution in the X direction when the fingertip FT is held upright relative to the operating surface 105A (left side) and when the fingertip FT is held flat relative to the operating surface 105A (right side). The double arrows indicate the range in which capacitance is detected. Note that while Figure 5C shows the capacitance distribution in the X direction, the capacitance distribution in the Y direction is similar.

[0061] As shown on the left side of Figure 5C, when the fingertip FT is held upright during operation, the capacitance distribution becomes steeper, and the range in which the fingertip FT can be detected narrows. As shown on the right side of Figure 5C, when the fingertip FT is laid flat and the palm is close to the operating surface 105A during operation, the capacitance distribution becomes flatter, and the range in which the fingertip FT can be detected widens.

[0062] Thus, the amount of variation in the measured XY coordinates due to noise is greater when the fingertip FT is held flat, as shown on the right side of Figure 5C, than when the fingertip FT is held upright, as shown on the left side of Figure 5C.

[0063] As explained above using Figures 5A, 5B, and 5C, the amount of variation in the measured XY coordinates due to noise varies depending on the distance from the fingertip FT to the operating surface 105A, the orientation of the fingertip FT, and other factors. Furthermore, the amount of variation also changes depending on the magnitude of the noise. In addition, the amount of variation increases when the fingertip FT is actually moved. When the amount of variation increases for any of the reasons, the measurement accuracy is poor, so the second pointer 112 is displayed larger. Conversely, when the amount of variation decreases, the second pointer 112 is displayed smaller. In this way, the position accuracy is represented by the size of the second pointer, allowing for intuitive feedback on position accuracy to the user.

[0064] Furthermore, as explained using Figure 5C, the amount of variation in the measured XY coordinates is greater when the fingertip FT is laid flat and the palm is close to the operating surface 105A than when the fingertip FT is held upright during operation. For this reason, if the amount of variation in the XY coordinates is greater than a predetermined threshold, the non-contact input device 100 may determine that the fingertip FT is not being held upright near the operating surface 105A and may display a message prompting the user to hold the fingertip FT upright near the operating surface 105A during operation.

[0065] <Calculation of the XY coordinates of the object (calculation of moving average)> As explained using Figures 5A to 5C, the XY coordinates calculated by the input sensor circuit 125A have a variable amount of change. Therefore, the input support application 150 calculates the moving average of the X coordinates acquired at multiple points in time and the moving average of the Y coordinates acquired at multiple points in time as the XY coordinates of the fingertip FT. The multiple points in time are the most recent multiple points in time for calculating the moving average, and the interval between points in time is several milliseconds.

[0066] The input support application 150 outputs a command to the OS 140 to display the image of the second pointer on the XY coordinates calculated as a moving average.

[0067] The XY coordinates of the fingertip FT detected by the non-contact input device 100 using the electrostatic sensor 120 represent, for example, the XY coordinates of the position with the largest capacitance within the area where the fingertip FT is located. Alternatively, the centroid of a figure formed by multiple points whose difference from the largest capacitance is less than or equal to a threshold may be considered as the XY coordinates of the fingertip FT. Furthermore, the X(Y) coordinates of the fingertip FT may be considered as the X(Y) coordinates of the vertex obtained by fitting the capacitances of the position with the largest capacitance and three points adjacent to it in the X (or Y) direction to a quadratic curve. Capacitance is inversely proportional to distance. Since capacitance and distance correspond one-to-one, capacitance can be used instead of distance. In addition, in the non-contact input device 100 of the present invention, it is not necessary to express the unit of capacitance in farads [F]. In the present invention, the value proportional to capacitance is called the Z value. The input sensor circuit 125A is designed to have a Z value of approximately 100 when the fingertip FT is in contact with the operating surface 105A, and a Z value of approximately 0 when the fingertip FT is not near the operating surface 105A.

[0068] The input assistance application 150 may stop calculating the moving average of the XY coordinates or the amount of change in the XY coordinates when in proximity mode or standby mode. In standby mode, the power to the display may also be turned off.

[0069] <Calculation of an index for the positional accuracy of an object using input support application 150> To indicate the magnitude of the error due to the variation in the XY coordinates measured by the non-contact input device 100, the input support application 150 calculates the standard deviation of the XY coordinates as an indicator of the positional accuracy of the object. The method for calculating the standard deviation of the XY coordinates will be described later using Figure 9D.

[0070] <Example of display for contactless input device 100> Figures 6A to 6F show examples of the display on the display 110 of the non-contact input device 100. Figures 6A and 6B show the display 110 and the operating surface 105A superimposed. The display on the display 110 is visible to the user through the operating surface 105A. Figures 6C to 6F show some of the images displayed on the display 110.

[0071] Figure 6A shows the display on the display 110 in standby mode. When the user's fingertip FT is not near the operating surface 105A, the messages "Touchless operation is possible" and "Please place your finger on the screen" are displayed, as shown in Figure 6A. This allows the user to guide their fingertip FT to the operating surface 105A even when using the contactless input device 100 for the first time, leading to smoother use thereafter.

[0072] Figure 6B shows an example of a message in proximity mode. In Figure 6B, the messages "The pointer will shrink according to the positional accuracy" and "If the pointer is large, move your finger closer or hold your finger up" are displayed. When the user brings their fingertip FT closer to the operating surface 105A, the second pointer 112 is displayed on the display 110. A pointer is a shape or symbol that indicates the current input position on the screen (image) displayed on the display 110.

[0073] The second pointer 112 is a shape that the input support application 150 displays on the OS 140. The input support application 150 also hides the first pointer (mouse pointer) 111 provided by the OS. The second pointer 112 may be a circular shape, an ellipse, or any other shape, in addition to the annular shape shown in Figure 6B. The input support application 150 increases the radius of the second pointer 112 as the amount of variation in the XY coordinates, which are an indicator of positional accuracy, increases, and decreases the radius of the second pointer 112 as the amount of variation in the XY coordinates decreases. In other words, the input support application 150 displays a second pointer 112 with a larger radius on the display 110 via the OS 140 as the amount of variation in the XY coordinates increases, and displays a second pointer 112 with a smaller radius on the display 110 via the OS 140 as the amount of variation in the XY coordinates decreases.

[0074] Here, using Figure 6C, we will describe in more detail the actions taken when the user brings their fingertip FT close to the operating surface 105A to select the GUI button for the number 6 in selection mode. In Figure 6C, the first pointer 111 is hidden, so only the second pointer 112 is shown. In selection mode, the annular second pointer 112 is displayed.

[0075] As shown on the left side of Figure 6C, the input assistance application 150 causes the OS 140 to display a second pointer 112 with a larger radius as the amount of variation in the XY coordinates increases. For example, when the fingertip FT is far from the operating surface 105A, the amount of variation in the XY coordinates increases.

[0076] Furthermore, as shown on the right side of Figure 6C, the input support application 150 displays a second pointer 112 on the OS 140 with a smaller radius as the amount of variation in the XY coordinates decreases. For example, when the fingertip FT is close to the operating surface 105A, the amount of variation in the XY coordinates decreases.

[0077] In other words, when operating the GUI button for the number 6, if the fingertip FT is far from the operating surface 105A, a second pointer 112 with a large radius is displayed, as shown on the left side of Figure 6C. Then, when the fingertip FT is brought closer to the operating surface 105A, the radius of the second pointer 112 decreases, and a second pointer 112 with a small radius is displayed, as shown on the right side of Figure 6C. Thus, when the fingertip FT is far from the operating surface 105A and the accuracy of the XY coordinates is low, the radius of the second pointer 112 is set to be large, and when the fingertip FT is close to the operating surface 105A and the accuracy of the XY coordinates is high, the radius of the second pointer 112 is set to be small. In this way, the non-contact input device 100 can provide intuitive feedback on positional accuracy to the user.

[0078] Figure 6D illustrates the operation when the user holds their fingertip FT close to the operating surface 105A to confirm the operation input of the number 6 GUI button in the confirmation mode. The second pointer 112A has a different display color from the second pointer 112 shown in Figure 6C. For this reason, the second pointer 112A is shown with a double line in Figure 6C. The input support application 150 intuitively indicates that the input mode has changed by changing the annular shape of the second pointer 112 to the arc of the second pointer 112A. Furthermore, the input support application 150 can emphasize the change in input mode by changing the second pointer from red (112) to yellow (112A). Note that the color of the second pointer is not limited to changing from red to yellow. The color of the second pointer (112, 112A) may be changeable in the settings screen described later. Alternatively, the brightness may be changed instead of the color, or both the color and brightness may be changed.

[0079] Figure 6D shows how the images change over time, starting from the top left, following the arrows, in the order of top right, bottom left, and bottom right.

[0080] In the decision mode, first, an arc-shaped second pointer 112A is displayed as shown in the upper left. When the fingertip FT is brought close to the operating surface 105A and held, the arc extends as shown in the upper right, and then the arc of the second pointer 112A extends further as shown in the lower left. After a first predetermined time T1 has elapsed since the fingertip FT has been stationary relative to the operating surface 105A, the second pointer 112A becomes a ring. The operation content is determined when the second pointer 112A becomes a ring. In other words, the operation content is determined by continuing to point the number 6 with the fingertip FT until the second pointer 112A becomes a ring. In this way, the non-contact input device 100 allows the user to perform a non-contact click operation instead of touching the operating surface 105A.

[0081] Once the operation is determined, the input support application 150 outputs a click event. The input support application 150 issues the same event as when a mouse is clicked, based on the non-contact operation input from the fingertip FT. The input support application 150 outputs the XY coordinates of the fingertip FT to the OS 140 and also outputs a click event. The OS 140 may also play a click event sound. This non-contact output of XY coordinates and the output of the click event are equivalent to touching the GUI button 110A on the operation surface 105A.

[0082] When the input support application 150 outputs a click event, it displays a second circular pointer 112B, which has a smaller radius than the ring, for a predetermined period of time (for example, about 1 to 2 seconds), as shown in the lower right of Figure 6D.

[0083] The circular second pointer 112B is displayed to the user for confirmation at the XY coordinates (moving average) of the fingertip FT when the operation input is confirmed. For example, the second pointer 112B is displayed for a predetermined time (approximately 1 to 2 seconds) and disappears after the predetermined time has elapsed. The display color of the second pointer 112B may be different from that of the second pointers 112 and 112A. The input support application 150 intuitively indicates that the operation input has been confirmed by changing the second pointer 112B to a small circle immediately after the central angle of the second pointer 112A (arc) reaches 360 degrees. Furthermore, the confirmation of the operation input can be emphasized by changing the color. Alternatively, the brightness may be changed instead of the color, or both the color and brightness may be changed.

[0084] Figures 6C and 6D illustrate an example where the first pointer 111 of OS140 is erased. However, as shown in Figure 6E, both the first pointer 111 (mouse pointer) of OS140 and the second pointer 112 (pointer for non-contact input device) provided by the input support application 150 may be displayed. When both the first pointer 111 and the second pointer 112 are displayed, the input support application 150 continues to output XY coordinates to OS140 in the selection mode and confirmation mode described later. The first pointer 111 performs the same function as a reticle that indicates the position of the XY coordinates with high precision.

[0085] <Display function menu> Figure 6F shows the function menu 112C. In selection mode, when the fingertip FT is brought close to the operating surface 105A and the radius of the second pointer 112 becomes small as shown on the right side of Figure 6C, and the XY coordinates of the fingertip FT remain stationary for a predetermined time, the input support application 150 displays the function menu 112C on the OS 140. This predetermined time is an example of a third predetermined time. Figure 6E shows, as an example, the function menu 112C in which click mode, drag mode, and pinch-in can be selected.

[0086] <Flowchart> Figures 7A to 7C, 8A to 8F, and 9A to 9F show examples of processing performed by the input support application 150 of the non-contact input device 100. Figures 7A to 7C show an example of the main processing, Figure 7D shows an example of a modified version of the processing shown in Figure 7C, and Figures 8A to 8F and 9A to 9F show examples of subroutine processing.

[0087] When the input support application 150 starts processing (see Figure 7A), it performs initial value setting (step S1). Step S1 is a subroutine, and its details will be described later using Figure 8A, but it sets initial values ​​such as the number of data points Nma for calculating the moving average.

[0088] The input support application 150 outputs a command to the OS 140 to erase the first pointer 111 (step S2). As a result, the first pointer 111 is erased from the display 110. Alternatively, the first pointer 111 (mouse pointer) may be displayed without performing the process in step S2 (see Figure 6E). In this case, both the first pointer 111 and the second pointer 112 will be displayed. In this case, the input support application 150 continues to output the XY coordinates of the fingertip FT to the OS. Since the position of the first pointer 111 and the center of the second pointer 112 coincide, the center of the second pointer 112 becomes easier to identify, even if it is a large circle.

[0089] The input support application 150 calculates the XY coordinates of the fingertip FT (step S3). Step S3 is a subroutine, and its details will be described later, but it calculates the moving average of the XY coordinates of the fingertip FT using the XY coordinates of the fingertip FT calculated at multiple points in time.

[0090] The input support application 150 determines whether its mode is the decision mode (step S4). Here, for the sake of clarity, we will first explain the case where the input support application 150 is in the decision mode.

[0091] If the input support application 150 determines that its mode is the determination mode (S4: YES), it determines whether the Z value is greater than Cz1 (step S5). If the Z value is greater than Cz1, the input support application 150 considers that the distance from the operating surface 105A to the fingertip FT is shorter than the first threshold Z1. The input support application 150 then determines from the Z value whether the fingertip FT is located within the determination region.

[0092] When the input support application 150 determines that the Z value is greater than Cz1 (S5: YES), it sets SelectionOffTime and ProximityOffTime to the current time (step S6). SelectionOffTime is updated to the time when it was determined that the Z value was greater than Cz1. Therefore, SelectionOffTime indicates the last time when it was determined that the Z value was greater than Cz1. Also, as will be described later, ProximityOffTime is updated when it is determined that the Z value is greater than Cz2. Therefore, ProximityOffTime indicates the last time when it was determined that the Z value was greater than Cz2.

[0093] The input support application 150 determines whether DecisionTH is 0 (S7). DecisionTH is a value that can be set by the user. If DecisionTH is 0, the input is confirmed immediately upon entering confirmation mode without displaying the arc cursor described later. On the other hand, if DecisionTH is not 0, the input is confirmed after displaying the arc cursor described later.

[0094] If the input support application 150 determines that DecisionTH is not 0 (S7: No), it outputs a command to the OS 140 to display an arc-shaped second pointer 112A on the XY coordinates (moving average: Xave, Yave) calculated in step S3 (step S8). The command output to the OS 140 in step S8 is a command to display an arc with a predetermined radius (fixed value) centered on the XY coordinates (moving average). The arc is an arc in which the moving end extends clockwise with respect to the fixed end, with the fixed end located at the 12 o'clock position and the moving end extending clockwise from 12 o'clock. The input support application 150 calculates the angle of the arc between the fixed end and the moving end using, for example, the following equation (1). Arc angle = 360 degrees × (current time - DecisionTime) / DecisionTH (1)

[0095] Here, DecisionTime is the time when the drawing of the arc began, and DecisionTH is the time required from the start of drawing the arc until a click event is output. A click event is an event that occurs when the operation content is determined by the operation input with the fingertip FT. The operation content is determined when the second pointer 112A changes from an arc to a ring. When the operation content is determined, the input support application 150 outputs a click event. The user can determine the operation content by bringing the fingertip FT close to the operation surface 105A within the determination area and holding the fingertip FT until the second pointer 112A extends into a ring.

[0096] The input support application 150 determines whether the condition Current Time - DecisionTime > DecisionTH is true (step S9). That is, it determines whether the elapsed time from the time (DecisionTime) when the system transitioned to decision mode has exceeded DecisionTH. This is to determine whether the operation content has been decided, and to determine whether to output a click event when the operation content has been decided. The elapsed time from the time (DecisionTime) when the system transitioned to decision mode is an example of a first predetermined time.

[0097] If the input support application 150 determines that the condition Current Time - DecisionTime > DecisionTH is not met (S9: NO), it returns the flow to step S3. As the processing in steps S3 to S8 is repeated, the arc extends.

[0098] When the input support application 150 determines that the condition Current Time - DecisionTime > DecisionTH is met (S9: YES), it proceeds with processing to output a click event. When Current Time - DecisionTime > DecisionTH is met, the arc cursor becomes a ring.

[0099] The input assistance application 150 erases the second pointer 112A (step S10).

[0100] The input support application 150 outputs the XY coordinates (moving average) calculated in step S3 to the OS 140, and then outputs a click event to the OS 140. Furthermore, the input support application 150 causes the OS 140 to play a click event sound (step S11). As a result, the click event sound is output from the speaker. Also, when the XY coordinates overlap with the GUI button 110A of the application program 160 and a click event is issued, the application program 160 executes the process indicated by the GUI button 110A.

[0101] The input support application 150 executes the process of step S11 described above when it determines that DecisionTH is 0 (S7:Yes). In other words, when it determines that "DecisionTH is 0" (S7:Yes) and when it determines that "current time - DecisionTime > DecisionTH" is true (S9:YES), the input support application 150 outputs the XY coordinates and the click event to the OS 140, and also causes the OS 140 to play the click event output sound.

[0102] The input support application 150 displays the second pointer 112B, centered on the XY coordinates (moving average) calculated in step S3, on the OS 140 for a predetermined time (step S12).

[0103] The center of the second pointer 112B is the XY coordinate (moving average) calculated in step S3. The predetermined time is, for example, about 1 to 2 seconds, and after the predetermined time has elapsed, the second pointer 112B becomes invisible.

[0104] The input support application 150 determines whether ReClickTH=0 is true (step S13). ReClickTH is the time (click re-output time) from the time the fingertip FT remains within the determination area after the click event is output until the second pointer 112A is displayed again. The user of the contactless input device 100 can set ReClickTH for the input support application 150 through the settings screen displayed on the display 110.

[0105] If the input support application 150 determines that ReClickTH=0 is true (S13:YES), it calculates the XY coordinates of the fingertip FT (step S14). Step S14 is a subroutine, similar to step S3, and its details will be described later, but it calculates the moving average of the XY coordinates of the fingertip FT using the XY coordinates of the fingertip FT calculated at multiple points in time.

[0106] The input support application 150 determines whether the Z value is greater than Cz1 (step S15). After the operation content is determined, the input support application 150 determines whether the fingertip FT remains within the determination area.

[0107] If the input support application 150 determines that the Z value is not greater than Cz1 (S15: NO), it sets it to selection mode (step S16). Then, it returns the flow to step S3. If the Z value ΔAD is not greater than Cz1, it means that the fingertip FT has moved from the decision area to the selection area or proximity area, etc. If the fingertip FT has moved from the decision area to the selection area or proximity area, etc., the processing in step S14 is provided to enable re-input.

[0108] If the input support application 150 determines in step S15 that the Z value is greater than Cz1 (S15: YES), it displays the second pointer 112B on the OS 140 at the XY coordinates (moving average) calculated in step S14 (step S17). The size of the circular second pointer 112B is a predetermined value.

[0109] If the click re-output time (ReClickTH) is set to 0, the input assistance application 150 will continue to display the second pointer 112B at the XY coordinate position of the fingertip FT without re-outputting the click event, unless it transitions from the confirmation mode to another mode.

[0110] When the input support application 150 finishes processing in step S17, it returns the flow to S14. If the operation content is determined by YES in step S7 or in step S9, and the second pointer 112B is displayed in step S12, and the fingertip FT remains within the determination area, the input support application 150 returns the flow to step S14 after displaying the second pointer 112B in step S17. In other words, it continues to display the second pointer 112B while updating its position. To determine the next operation content after the operation content has been determined once, the fingertip FT should be moved out of the determination area once.

[0111] <s5:no> Furthermore, in step S5, if the input support application 150 determines that the Z value is not greater than Cz1 (S5: NO), it determines whether the condition Current Time - SelectionOffTime > SelectionOffTH is true (step S18). That is, when the fingertip FT is continuously located outside the selection area, it determines whether the elapsed time since the last time (SelectionOffTime) when the fingertip FT was determined to be located inside the selection area exceeds SelectionOffTH. This elapsed time is an example of the fourth predetermined time.

[0112] SelectionOffTH is the time it takes to transition from confirmation mode to selection mode when the fingertip FT is moved to the selection area. The user of the contactless input device 100 can set SelectionOffTH for the input support application 150 through the settings screen displayed on the display 110. The time when the fingertip FT was last determined to be within the confirmation area is the most recent time when the fingertip FT was determined to be within the confirmation area.

[0113] If the input support application 150 determines that the condition Current Time - SelectionOffTime > SelectionOffTH is not met (S18: NO), it determines whether the Z value is greater than Cz2 (step S19). If the Z value is greater than Cz2, the input support application 150 assumes that the fingertip FT is remaining in the first region. If Z is greater than Cz2 (S19: YES), the flow returns to step S7. As a result, operation in decision mode continues. The Z value does not use a moving average, and noise may cause a value smaller than the actual value to be measured. In other words, even if the fingertip FT is located in the decision region, the condition Z > Cz1 may not be met (S5: NO). For this reason, even if the Z value is not greater than Cz1, if the Z value is greater than Cz2, the drawing of the second pointer 112A continues for a while (during SelectionOffTH). On the other hand, if the input support application 150 is not greater than Cz2 (S19: NO), it sets the application to selection mode and sets the current time to SelectionTime (step S20). If the Z value is not greater than Cz2, the fingertip FT is considered not to be in the selection area.

[0114] Furthermore, if the input support application 150 determines that the condition Current Time - SelectionOffTime > SelectionOffTH is met (S18: YES), it sets the mode of the input support application 150 to selection mode and sets SelectionTime to the current time (step S20). If the state in which the Z value is not greater than Cz1 (step S5) continues for longer than SelectionOffTH, it is determined that the fingertip FT is not in the selection area.

[0115] The input assistance application 150 causes the OS 140 to hide the second pointer 112A (step S21).

[0116] The input support application 150 displays the second pointer 112 on the OS 140 (step S22). The second pointer 112 is the second pointer for the selection mode shown in Figure 6C. Before displaying the second pointer 112 on the OS 140, the input support application 150 hides the second pointer 112A. The center of the second pointer 112 is the XY coordinates (moving average) of the fingertip FT, and the radius of the second pointer 112 is the radius corresponding to the amount of change in the XY coordinates. In step S22, when displaying the second pointer 112 on the OS 140, the input support application 150 outputs the position of the second pointer 112 corresponding to the XY coordinates (moving average) of the fingertip FT to the OS 140. The input support application 150 also adjusts the size of the second pointer 112 to a size corresponding to the amount of change in the XY coordinates.

[0117] The input support application 150 performs a process to determine whether or not to display the function menu (step S23). Since the process to determine whether or not to display the function menu is a subroutine process, its details will be described later. After completing the process in step S23, the input support application 150 returns the flow to step S3.

[0118] <s12:no> If the input support application 150 determines in step S13 that ReClickTH=0 is not true (S13:NO), it sets ClickTime to the current time (step S24). ClickTime is the time when the click event was output.

[0119] The input support application 150 determines whether the condition Current Time - ClickTime > ReClickTH is true (step S25).

[0120] The input support application 150 sets DecisionTime to the current time (step S26) when Current Time - ClickTime > ReClickTH is true (S25: YES). Then it returns the flow to step S3. Therefore, DecisionTime becomes the time when it returns to step S3. When the fingertip FT is kept in the decision area, a gradually lengthening arc is drawn again. The time from when a click event is output until the drawing of the second pointer 112A (arc) starts again is the "determined time" in step S12 plus ReClickTH. From the start of drawing the second pointer 112A (arc) until the second pointer 112A becomes a ring, DecisionTimeTH takes time. A click event is output each time the "determined time" in step S12 plus ReClickTH and DecisionTimeTH has elapsed. In other words, the "determined time" in step S12 plus ReClickTH and DecisionTimeTH becomes the second predetermined time.

[0121] If the input support application 150 determines in step S25 that the condition Current Time - ClickTime > ReClickTH is not met (S25: NO), it calculates the XY coordinates of the fingertip FT (step S27). Step S27 is a subroutine, similar to step S3, and its details will be described later, but it calculates a moving average of the XY coordinates of the fingertip FT using the XY coordinates of the fingertip FT calculated at multiple points in time.

[0122] The input support application 150 determines whether the Z value is greater than Cz1 (step S28). This is to determine whether the fingertip FT is located within the determination area.

[0123] If the input support application 150 determines that the Z value is greater than Cz1 (S28: YES), it displays the second pointer 112B at a position centered on the XY coordinates of the fingertip FT (step S29). If the input support application 150 remains in confirmation mode, it displays the second pointer 112B on the OS 140 until the click re-output time (ReClickTH) has elapsed.

[0124] In step S27, if the input support application 150 determines that the Z value is not greater than Cz1 (S28: NO), it sets the mode of the input support application 150 to selection mode and sets the current time to SelectionTime (step S30). When the fingertip FT moves from the determination area to the selection area, it immediately transitions to selection mode. Users who want to perform the next operation input as quickly as possible can switch to selection mode by slightly moving the fingertip FT away from the operation surface 105A. After completing the processing in step S30, the input support application 150 returns the flow to step S3.

[0125] <s4:no> Furthermore, if the input support application 150 determines in step S4 that it is not in decision mode (S4: NO), it determines whether the Z value is greater than Cz1 (step S4A). In other words, the input support application 150 determines whether the fingertip FT is located within the decision area. If the Z value is greater than Cz1, the fingertip FT is located within the decision area.

[0126] When the input support application 150 determines that the Z value is greater than Cz1 (S4A: YES), it sets SelectionOffTime and ProximityOffTime to the current time (step S5A). SelectionOffTime is updated to the time when it was determined that the Z value was greater than Cz1, so SelectionOffTime indicates the last time when the Z value was determined to be greater than Cz1. Also, as will be described later, ProximityOffTime is updated when it is determined that the Z value is greater than Cz2. Therefore, ProximityOffTime indicates the last time when the Z value was determined to be greater than Cz2.

[0127] The input support application 150 sets DecisionTime to the current time (step S6A). The input support application 150 outputs a click event when the time spent in decision mode reaches DecisionTH. In step S4, it was determined that the user was not in decision mode, and in step S4A, it was determined that the Z value was greater than Cz1, so the time when step S6A is executed is the time when the user entered decision mode. In other words, the DecisionTime set in step S6A is the time when the user entered decision mode.

[0128] The input support application 150 sets its mode to the decision mode (step S7A). Furthermore, the input support application 150 clears the second pointer 112 for the selection mode (step S8A). After completing the process in step S8A, the flow proceeds to step S7.

[0129] The flow progresses from step S4:NO to S4A:YES when the fingertip FT enters the decision area from outside the decision area. In this way, when the fingertip FT approaches the operation surface 105A, the system immediately transitions to decision mode. On the other hand, when the fingertip FT moves away from the operation surface 105A, the system does not immediately change mode. As mentioned above, when the Z value becomes small (when the fingertip FT is measured to be moving away from the operation surface 105A), the system does not immediately change mode to stabilize the operation. Conversely, by not using a moving average for the Z value and immediately transitioning to decision mode when the fingertip FT approaches the operation surface 105A, quick operation is achieved. Note that when the system enters decision mode due to noise, the fingertip FT is not in the decision area but is near the decision area (selection area). Therefore, the user does not perceive it as a malfunction. This achieves both prevention of a decrease in response speed and prevention of incorrect input.

[0130] <s4a:no> Furthermore, if the input support application 150 determines in step S4A that the Z value is not greater than Cz1 (S4A: NO), it determines whether the mode of the input support application 150 is the selection mode (step S4B).

[0131] In selection mode, the input support application 150 determines whether the Z value is greater than Cz2 (step S5B). That is, the input support application 150 determines whether the fingertip FT is located within the selection area.

[0132] If the input support application 150 determines that the Z value is greater than Cz2 (S5B: YES), it sets ProximityOffTime to the current time (step S6B). ProximityOffTime is the time when the fingertip FT was last determined to be located within the selection area while it is continuously located within the selection area. When the fingertip FT is located within the selection area, ProximityOffTime is updated, so ProximityOffTime becomes the most recent time when the fingertip FT was determined to be located within the selection area.

[0133] The input support application 150 executes a second pointer radius setting process (step S7B) which sets the radius of the second pointer 112 according to the amount of change in the XY coordinates. The second pointer radius setting process is a subroutine process and will be described in detail later, but in selection mode, the radius of the second pointer 112 (see Figure 6C) is set according to the amount of change in the XY coordinates. After completing the process in step S7B, the input support application 150 proceeds to step S21 and displays the second pointer 112 on the OS 140.

[0134] <s5b:no> If the input support application 150 determines in step S5B that the Z value is not greater than Cz2 (S5B: NO), it determines whether the condition Current Time - ProximityOffTime > ProximityOffTH is true (step S8B). That is, when the fingertip FT is continuously located within the selection area, it determines whether the elapsed time since the last time (ProximityOffTime) when the fingertip FT was determined to be within the selection area exceeds ProximityOffTH. ProximityOffTH is the time from selection mode to proximity mode. The user of the contactless input device 100 can set ProximityOffTH for the input support application 150 through the settings screen displayed on the display 110.

[0135] If the input support application 150 determines that the condition Current Time - ProximityOffTime > ProximityOffTH does not hold true (S8B: NO), it executes a second pointer radius setting process (step S9B) to set the radius of the second pointer 112 according to the amount of change in the XY coordinates. The second pointer radius setting process is a subroutine process and will be described in detail later, but in selection mode, the radius of the second pointer 112 (see Figure 6C) is set according to the amount of change in the XY coordinates. After completing the process in step S9B, the input support application 150 proceeds to step S21 and displays the second pointer 112 on the OS 140. The Z value does not use a moving average, and due to noise, a smaller value than the actual value may be measured. However, the condition where Z > Cz2 does not hold true, even though the actual position of the fingertip FT is within the selection area, is resolved in a shorter time than ProximityOffTH. Therefore, even if a small Z value is measured due to noise while the second pointer 112 is being displayed, the display of the second pointer 112 will continue. On the other hand, even if the fingertip FT is actually moved to the proximity area, the display of the second pointer 112 will continue for a while. Most users will not perceive a delay in response even if the time it takes for the display of the second pointer 112 to disappear is delayed. Therefore, the processing from S8B:NO to S9B can stabilize the operation without reducing the perceived response speed.

[0136] If the input support application 150 determines in step S8B that the condition Current Time - ProximityOffTime > ProximityOffTH is true (S8B: YES), the mode of the input support application 150 is set to proximity mode (step S10B).

[0137] The input support application 150 displays an image and message for proximity mode (see Figure 6B) on the OS 140 (step S11B). As a result, the image and message for proximity mode are displayed on the display 110, as shown in Figure 6B. After completing the processing in step S11B, the input support application 150 proceeds to step S3.

[0138] <s4b:no> Furthermore, if the input support application 150 determines in step S4B that its mode is not selection mode (S4B: NO), it determines whether the Z value is greater than Cz2 (step S4C). In other words, the input support application 150 determines whether the fingertip FT has entered the selection area again.

[0139] The input support application 150 determines that the Z value is greater than Cz2 (S4C: YES) and sets ProximityOffTime to the current time (step S5C). As mentioned above, if Z > Cz1 (S5 or S4A) is YES, or if Z > Cz2 (S5B or S4C) is YES, ProximityOffTime is updated to the current time. Therefore, the last time when Z was greater than Cz2 is stored as ProximityOffTime.

[0140] The input support application 150 sets its mode to selection mode and sets the current time to SelectionTime (step S6C).

[0141] The input support application 150 performs a process to set the radius of the second pointer 112 to an initial value (step S7C). The process of setting the radius of the second pointer 112 to an initial value is a subroutine process and will be described in detail later, but it is a process that sets the radius of the second pointer 112 to an initial value, rather than a radius corresponding to the amount of change in the XY coordinates. After completing the process in step S7C, the input support application 150 proceeds to step S21 and displays the second pointer 112 on the OS 140. Note that although the fingertip FT is actually in the proximity area, noise may cause the Z value to be measured as larger than it actually is, resulting in selection mode. However, since selection mode is entered when the fingertip FT is relatively close, most users do not perceive it as a malfunction. Conversely, if the fingertip FT is in a distant position (standby area), selection mode will not be entered even if there is noise at a normal level.

[0142] <s4c:no> If the input support application 150 determines in step S4C that the Z value is not greater than Cz2 (S4C: NO), it determines whether the mode of the input support application 150 is proximity mode (step S4D).

[0143] If the input support application 150 determines that its mode is proximity mode (S4D: YES), it determines whether the Z value is greater than Cz3 (step S5D). If the Z value is greater than Cz2, the input support application 150 considers that the distance from the operating surface 105A to the fingertip FT is shorter than the third threshold Z3.

[0144] If the input support application 150 determines that the Z value is greater than Cz3 (S5D:YES), it proceeds to step S11B. As a result, in step S11B, an image and message for proximity mode are displayed on the display 110, as shown in Figure 6B.

[0145] <s5d:no> If the input support application 150 determines in step S5D that the Z value is not greater than Cz3 (S5D: NO), it sets the mode of the input support application 150 to standby mode (step S6D). If the Z value is not greater than Cz3, the input support application 150 considers that the distance from the operating surface 105A to the fingertip FT is longer than the third threshold Z3.

[0146] The input support application 150 displays an image and message for standby mode on the OS 140 (step S7D). As a result, the image and message for standby mode are displayed on the display 110. After completing the processing in step S7D, the input support application 150 proceeds to step S3. Note that the power to the display may be turned off in standby mode (S7D' in Figure 7D).

[0147] <s4d:no> If the input support application 150 determines in step S4D that its mode is not proximity mode (S4D: NO), it then determines whether the Z value is greater than Cz3 (step S4E). The input support application 150 then determines whether the fingertip FT has entered the proximity area again.

[0148] If the input support application 150 determines that the Z value is greater than Cz3 (S4E: YES), it sets the mode of the input support application 150 to proximity mode (step S5E). The input support application 150 then proceeds to step S11B, where it displays an image and message for proximity mode.

[0149] Furthermore, if the input support application 150 determines in step S4E that the Z value is not greater than Cz3 (S4E: NO), it proceeds to step S7D. In this case, since the input support application 150 is in standby mode, an image and message for standby mode are displayed on the display 110 in step S7D.

[0150] This completes the main flow. The input assistance application 150 will repeat the main flow. Next, we will describe each subroutine.

[0151] <Figure 8A: Initial Value Setting> The input support application 150 sets its mode to standby mode and sets the flag null to YES (step S101).

[0152] The input support application 150 sets Nma to a predetermined integer (step S102). Nma is the calculated number of data points used when calculating the moving average, and is calculated as Nma = int(MATime / Nmea). A predetermined integer is obtained as int(MATime / Nmea). int(MATime / Nmea) represents the integer obtained by truncating the decimal part of MATime / Nmea. MATime is the time used for the moving average (time from the start to the end), and Nmea is the measurement interval. That is, Nma = int(MATime / Nmea). The user of the non-contact input device 100 can set MATime for the input support application 150 through the setting screen displayed on the display 110.

[0153] The input support application 150 compares Nma and Ndev and determines which is larger (step S103). Ndev is the number of data points used when calculating the position accuracy index.

[0154] The input support application 150 sets the array size Num to the larger of Nma and Ndev (steps S104, S105). In other words, it sets the array size Num to the larger of either "the number of data points required to calculate the moving average" or "the number of data points required to calculate the positional accuracy index". Since the number of elements in the array is determined, the array may be declared at this stage. After completing the processing in step S104, the input support application 150 terminates the initial value setting subroutine.

[0155] <Figure 8B: Coordinate Calculation> The input support application 150 obtains the XY coordinates and Z value from the input sensor circuit 125A (step S111).

[0156] The input support application 150 obtains the current time from the OS 140 (step S112). The "current time" used in the main loop described above is the value obtained in step S112.

[0157] The input support application 150 updates the array of X coordinates (step S113). Step S113 is a subroutine process and will be described in detail later, but in step S113, the input support application 150 removes the oldest value and adds the latest value from the array used to calculate the moving average of X coordinates and the index of positional accuracy.

[0158] The input support application 150 updates the Y-coordinate array (step S114). Step S113 is a subroutine process and will be described in detail later, but in step S114, the input support application 150 removes the oldest value and adds the latest value from the array used to calculate the moving average of the Y-coordinates and the position accuracy index.

[0159] The input support application 150 calculates a moving average from the X coordinate array updated in step S113 (step S115).

[0160] The input support application 150 calculates the array moving average of the Y coordinates updated in step S114 (step S116).

[0161] The input support application 150 determines whether the Z value obtained in step S111 is greater than Cz3 (step S117). If the Z value is not greater than Cz3, the Z value falls within the waiting area, and the reliability of the latest XY coordinates obtained in step S111 and the XY coordinates obtained in step S111 before that is low. Therefore, the coordinate values ​​are not suitable for calculating the moving average of the XY coordinates, and the process in step S117 is included.

[0162] The input support application 150 determines that the Z value obtained in step S111 is greater than Cz3 (S117: YES), and sets the flag null to NO (step S118). The flag null is set to NO because the latest XY coordinates are suitable coordinate values ​​for calculating the moving average of the XY coordinates and the index of positional accuracy.

[0163] The input support application 150 determines that the Z value obtained in step S111 is not greater than Cz3 (S117: NO), and sets the flag null to YES (step S119). The flag null is set to YES because the latest XY coordinates are not suitable coordinate values ​​for calculating the moving average of the XY coordinates.

[0164] This completes the coordinate calculation subroutine.

[0165] <Figure 8C: Updating the X-coordinate array> Figure 8C shows the details of the subroutine process that updates the array of X coordinates in step S113 of Figure 8B. Hereinafter, multiple X coordinates acquired at multiple points in time are distinguished as X(i). The time at which the multiple X coordinates X(i) were acquired is different from each other, and they are multiple X coordinates acquired continuously by the non-contact input device 100 at a predetermined sampling period. i = 1 to Num, and X(1) is the most recent X coordinate.

[0166] The input assistance application 150 determines whether the flag null is YES (step S121X). This is to determine whether the latest X coordinate is appropriate.

[0167] If the input support application 150 determines that the flag null is YES (S121X:YES), it performs a subroutine process to create the X coordinate by substituting the non-measured value into X(i).

[0168] The input support application 150 selects Num-1 X coordinates (X(2) to X(Num)) from i = 2 to Num one by one and assigns a non-measured value to them (step S122X). A non-measured value is a value that has not been measured and is a dummy value. Here, as an example, 0xFFFF is used as the non-measured value. After completing the subroutine processing including step S122X, the input support application 150 proceeds to step S123X. Note that the non-measured value can be any value that has never been measured and may be a value other than 0xFFFF.

[0169] The input assistance application 150 assigns the latest X coordinate to X(1) (step S123X). In this way, the input assistance application 150 updates Num X coordinates X(1) to X(Num).

[0170] Furthermore, if the input support application 150 determines that the flag null is NO (S121X: NO), it selects Num-1 X coordinates (X(1) to X(Num-1)) from i 1 to Num-1 one by one, increments the i number by 1, and moves to the X coordinates (X(2) to X(Num)) (step S124X).

[0171] After completing the processing in step S124X, the input support application 150 proceeds to step S123X. By adding the X coordinate X(1) to the X coordinates (X(2)~X(Num)) obtained in step S124X, the input support application 150 updates the array of X coordinates X(1)~X(Num).

[0172] <Figure 8D: Obtaining the latest Num Y coordinates> Figure 8D shows the details of the subroutine process that updates the array of Y coordinates in step S114 of Figure 8B. Hereinafter, multiple Y coordinates are distinguished as Y(i). The times at which the multiple Y coordinates Y(i) were acquired are different from each other, and they are multiple Y coordinates that the non-contact input device 100 acquired continuously at a predetermined sampling period. The time at which each of the multiple Y coordinates Y(i) was acquired is equal to the time at which each of the multiple X coordinates X(i) was acquired. i = 1 to Num, and Y(1) is the most recent Y coordinate.

[0173] The steps S121Y to S124Y shown in Figure 8D are the processes that the input support application 150 executes for the Y coordinate Y(i), and are the same as the steps S121X to S124X shown in Figure 8C for the X coordinate X(i), but replaced with processes for the Y coordinate Y(i). Therefore, the explanation of Figure 8D is omitted here. Alternatively, instead of the processes S121X to S123X and S121Y to S123Y, the oldest value in the array may be replaced with the newest value. In this case, it becomes unnecessary to copy each element of the array. However, an index to identify the element containing the oldest value and an index to identify the element containing the newest value will be required. In addition, the loop processing for calculating the moving average and the amount of change, which will be described later, will also need to be modified.

[0174] <Figure 8E: Moving average of X coordinates> Figure 8E shows the details of the subroutine process that calculates the moving average of the X coordinate in step S115 of Figure 8B.

[0175] The input support application 150 resets the integrated value Xacc of the X coordinates to 0 (step S131X).

[0176] The input support application 150 executes the steps S132X, 133X, and 134X, which are included in the subroutine that performs the integration of X coordinates. In this subroutine, the integration of X(i) is performed one by one from 1 to Nma.

[0177] The input support application 150 determines whether X(i) is 0xFFFF (step S132X). 0xFFFF means that there is no measured value of the Z value used to calculate the X coordinate.

[0178] If the input support application 150 determines that X(i) is 0xFFFF (S132X:YES), it updates the cumulative value Xacc to Xacc+X(1) (step S133X). In other words, if dummy data is stored in X(i), the latest X coordinate (X(1)) is used to calculate the cumulative value. Therefore, the calculation Xacc = Xacc + X(1) is performed.

[0179] Furthermore, if the input support application 150 determines in step S132X that X(i) is not 0xFFFF (S132X: NO), it updates the accumulated value Xacc to Xacc + X(i) (step S134X). In other words, it accumulates the previously measured X coordinates (X(i)) in order. Therefore, it performs the calculation Xacc = Xacc + X(i).

[0180] The input support application 150 repeatedly executes the processes in steps S132X, 133X, and 134X for i in X(i) from 1 to Nma, then finishes processing the subroutine that performs the integration of the X coordinates, and proceeds to step S135X.

[0181] The input support application 150 calculates the moving average of the X coordinate by dividing the accumulated value Xacc by Nma (step S135X). That is, the moving average of the X coordinate Xave = Xacc / Nma.

[0182] <Figure 8F: Moving average of Y coordinates> Figure 8F shows the details of the subroutine process that calculates the moving average of the Y coordinate in step S116 of Figure 8B.

[0183] The steps S131Y to S135Y shown in Figure 8F are the processes that the input support application 150 executes for the Y coordinate Y(i), and are the same as the steps S131X to S135X shown in Figure 8E for the X coordinate X(i), but replaced with processes for the Y coordinate Y(i). For this reason, the explanation of Figure 8F is omitted here.

[0184] <Figure 9A: Process to set the radius of the second pointer 112 to its initial value> Figure 9A shows an example of the process of setting the radius of the second pointer 112 to an initial value in step S7C.

[0185] The input support application 150 sets the initial value of the radius Radius of the second pointer 112 to its maximum value Rmax (step S141A). That is, Radius = Rmax. The user of the contactless input device 100 can set Rmax for the input support application 150 through the settings screen displayed on the display 110.

[0186] <Figure 9B: Process to set the radius of the second pointer 112 to its initial value> Figure 9B shows another example of the process of setting the radius of the second pointer 112 to an initial value in step S7C.

[0187] The input support application 150 calculates the amount of variation D of the XY coordinates as an indicator of positional accuracy (step S141B). Step S141B is a subroutine process, and its details will be described later using Figure 9E. As an example, the input support application 150 calculates the amount of variation D when it enters selection mode. The input support application 150 sets the maximum value Rmax of the radius of the second pointer 112 to a value obtained by multiplying the amount of variation D of the XY coordinates by a constant (step S142B). That is, Rmax = D × constant. The predetermined constant is a value that is set in advance.

[0188] The input support application 150 sets the radius Radius of the second pointer 112 to the maximum value Rmax set in step S142B (step S143B). That is, Radius = Rmax = D × constant. Note that when performing the process shown in Figure 9B, the input support application 150 does not perform the process that allows the user to set the maximum value Rmax.

[0189] <Figure 9C: Process to set the radius of the second pointer 112> Figure 9C shows an example of the radius setting process for the second pointer in steps S7B and S9B.

[0190] The input support application 150 calculates the amount of variation D of the XY coordinates as an indicator of positional accuracy (step S141C). Step S141C is a subroutine process, and its details will be described later using Figure 9E. As an example, the input support application 150 calculates the amount of variation D when the selected mode is activated.

[0191] The input support application 150 calculates the radius Radius of the second pointer 112 using the variation D obtained in step S141C (step S142C). The radius Radius = Rmin + (Rmax - Rmin) × (D / Rmax). Rmin is the minimum value of the radius of the second pointer 112 and is a predetermined value as an example.

[0192] The input support application 150 determines whether the radius Radius calculated in step S142C is greater than the maximum value Rmax (step S143C).

[0193] If the input support application 150 determines that the radius Radius is greater than the maximum value Rmax (S143C: YES), it sets the radius Radius to the maximum value Rmax (step S144C). That is, Radius = Rmax.

[0194] Furthermore, if the input support application 150 determines that the radius Radius is not greater than the maximum value Rmax (S143C:NO), it uses the calculated radius Radius as is.

[0195] As described above, the input support application 150 calculates the amount of variation D when it enters selection mode, for example. When the second pointer 112 is displayed in selection mode, the size of the second pointer 112 gradually changes, making it easier to see the second pointer 112, which is sized according to the positional accuracy.

[0196] For example, the input support application 150 may calculate the fluctuation amount D when it enters proximity mode. By calculating the position accuracy before displaying the second pointer 112 in selection mode, the second pointer 112 can be displayed with a size corresponding to the position accuracy from the moment it starts to be displayed.

[0197] <Figure 9D: Calculation process for the amount of change D in the XY coordinates> Figure 9D shows an example of the calculation process for the amount of change D in the XY coordinates. Figure 9D shows an example of the calculation process for the amount of change D in step S141C of Figure 8C.

[0198] The input support application 150 resets the change in the X coordinate DX and the change in the Y coordinate DY (step S151A). That is, DX = 0 and DY = 0.

[0199] The input support application 150 performs a subroutine process to calculate the cumulative amount of variation D. This subroutine includes steps S152A, S153A, and S154A, and is performed by setting i from 1 to Ndev for the X coordinate X(i) and Y coordinate Y(i). In other words, the input support application 150 calculates the amount of variation D using the most recent Ndev XY coordinates.

[0200] The input support application 150 determines whether the X coordinate X(i) is 0xFFFF (step S152A).

[0201] When the input support application 150 determines that the X coordinate X(i) is 0xFFFF (S152A:YES), it adds a constant to the change in the X coordinate DX and the change in the Y coordinate DY (step S153A). That is, DX = DX + constant and DY = DY + constant.

[0202] Furthermore, if the input support application 150 determines that the X-coordinate X(i) is not 0xFFFF (S152A: NO), it adds the square of the difference between the X-coordinate fluctuation amount DX and the Y-coordinate fluctuation amount DY and the moving average Xave of the X-coordinate and the moving average Xave of the Y-coordinate (step S154A). That is, DX = DX + (X(i) - Xave) 2 DY = DY + (Y(i) - Yave) 2 This is the result.

[0203] The input support application 150 includes steps S152A, S153A, and S154A, where, for the X coordinate X(i) and Y coordinate Y(i), if i is set from 1 to Ndev, the flow proceeds to step S155A.

[0204] The input support application 150 uses DX and DY calculated in the subroutine process to calculate the amount of change D of the XY coordinates according to the following equation (2) (step S155A).

[0205]

number

[0206] The variation D is then determined. By determining DX and DY in step S154A and the variation D in step S155A, the standard deviation of the XY coordinates can be determined as an indicator of positional accuracy.

[0207] The input support application 150 determines, based on the variation amount D, whether the fingertip FT is operating with the finger upright and close to the operating surface 105A (step S156A). If the variation amount D is large, the fingertip FT is either far from the operating surface 105A, the finger is not extended, or the finger is flat. If the variation amount D is greater than the palm threshold (S156A: YES), a warning is displayed (step S157A). The warning is a message such as, "Please operate with your finger upright and close to the operating surface." In addition, the entire screen may be turned red along with the warning.

[0208] Furthermore, if the fingertip FT is suddenly brought close to the operating surface 105A from standby mode, the fluctuation amount D will be a large value, resulting in poor positional accuracy. On the other hand, if the fingertip FT is kept close to the operating surface 105A, the dummy data substituted in step S122X will gradually be replaced by measurement data, and the fluctuation amount D will decrease. In this case, the positional accuracy will gradually improve. The fluctuation amount D may also be obtained by the process shown in Figure 9E.

[0209] <Figure 9E: Modified example of the calculation process for the variation D of the XY coordinates> Figure 9E shows an example of the calculation process for the amount of change D in the XY coordinates. Figure 9D shows an example of the calculation process for the amount of change D in step S141C of Figure 8C.

[0210] The process shown in Figure 9E is a modified version of the process shown in Figure 9D, specifically step S155A. Therefore, the processes from steps S151B to S154B in Figure 9E are identical to the processes from steps S151A to S154A in Figure 9D. For this reason, only the process of step S155B will be explained here.

[0211] The input support application 150 uses DX and DY calculated in the subroutine process to calculate the amount of change D of the XY coordinates according to the following equation (3) (step S155A).

[0212] D = DX + DY (3)

[0213] In Figure 9E, the variation D is not divided by Ndev. For example, if Ndev is a constant, the sum of DX and DY (without dividing by Ndev) may be used as the variation D. Also, the variation D may be either DX or DY, rather than the variation D of the XY coordinates. For example, it may be either the variation of the X coordinate DX / Ndev, or the variation of the Y coordinate DY / Ndev.

[0214] <Figure 9F: Determination process for whether or not to display the menu> Figure 9F shows an example of the process for determining whether or not to display a menu. Figure 9F shows the process for determining whether or not to display a menu in step S23.

[0215] Here, Nmenu is the number of data points corresponding to the time slot for determining whether or not to display the menu. Nmenu may be defined during the design phase, or it may be set by the user from the settings screen. Dmenu is the distance corresponding to the threshold amount of movement of the X and Y coordinates.

[0216] The input support application 150 performs an X-coordinate determination process (determining X) while incrementing the i-number of X(i) from 2 to Nmenu by 1. The X-coordinate determination process is a subroutine process.

[0217] The input support application 150 determines whether the absolute value of X(i)-X(1) is less than Dmenu (step S161). X(1) is the latest X coordinate.

[0218] The input support application 150 performs subroutine processing on the Y coordinate if the absolute values ​​of the differences between X(2)~X(Nmenu) and X(1) are all less than Dmenu (all S161:YES).

[0219] The input support application 150 performs a Y-coordinate determination process (determining Y) while incrementing the i-number of Y(i) from 2 to Nmenu by 1. The Y-coordinate determination process is a subroutine process.

[0220] The input support application 150 determines whether the absolute value of Y(i)-Y(1) is less than Dmenu (step S162). Y(1) is the latest Y coordinate.

[0221] The input support application 150 proceeds to step S163 if the absolute values ​​of the differences between Y(2)~Y(Nmenu) and Y(1) are all less than Dmenu (all S162:YES).

[0222] The input support application 150 determines whether the condition Current Time - SelectionTime > SelectionTH holds true (step S163). In other words, if the elapsed time since entering selection mode exceeds a predetermined time, the flow proceeds to step S164.

[0223] The input support application 150 displays a menu (step S164). The menu is, for example, a function menu 112C (see Figure 6E). If a finger is moving (S161: NO, or S162: NO) or immediately after entering selection mode (S163: NO), the function menu is not displayed.

[0224] The menu is displayed in selection mode when N menus of XY coordinates have not moved (are stationary). In other words, in selection mode, the input support application 150 causes the OS 140 to display the menu when the XY coordinates of the fingertip FT remain stationary for the duration that N menus of XY coordinates are acquired. The duration that N menus of XY coordinates are acquired is an example of a third predetermined time.

[0225] <Settings screen> Figure 10 shows an example of a settings screen. The input support application 150 causes the OS 140 to display the settings screen shown in Figure 10 on the display 110. The settings screen is an image representing a screen where various settings can be entered.

[0226] The settings screen shown in Figure 10 includes five text boxes for setting mode switching thresholds, a message to display on the display 110 in standby mode and proximity mode, a text box for inputting the moving average of the position of the second pointer 112, the click event issuance time, the minimum and maximum values ​​of the radius of the second pointer 112, and the click re-output time. The threshold for proximity mode corresponds to the Z value Cz3, the threshold for selection mode corresponds to the Z value Cz2, and the threshold for decision mode corresponds to the Z value Cz1.

[0227] Figure 11 shows an example of a folder structure. The folders are stored inside memory 132 (see Figure 2), and the input support application 150 can access them.

[0228] Figure 11 shows the audio file (Click.wav) and background image files (Proximity.png, Waiting.png) that can be selected in the settings screen. Note that, although the audio file and background image file are shown in the same folder here, the folders for the audio file and background image file can be separate.

[0229] <Flowchart of the setup process> Figure 12 is a flowchart showing an example of the setup process. When "Settings" is selected from the menu in the task tray, the input support application 150 launches the setup program and executes the process shown in Figure 12.

[0230] The input support application 150 displays the settings screen on the display 110 via the OS 140 (step S201). The user can input various values ​​on the settings screen shown in Figure 10.

[0231] The input support application 150 determines whether the apply button has been clicked (step S202).

[0232] When the input support application 150 determines that the Apply button has been clicked (S202:YES), it sets various values ​​(step S203). Specifically, the input support application 150 inputs the value from the proximity mode setting box into ProximityTH, the value from the selection mode setting box into SelectionTH, the value from the decision mode setting box into DecisionTH, the value for the OFF determination to proximity mode into ProximityOffTH, and the value for the OFF determination to selection mode into SelectionOffTH. The input support application 150 also inputs the click re-output time value into ReClickTH, the second pointer moving average time into MATime, the click event issuance time value into DecisionTimeTH, the minimum radius value into Rmin, the maximum radius value into Rmax, saves a message for standby mode, and saves a message for proximity mode.

[0233] Furthermore, if the input support application 150 determines in step S202 that the Apply button has not been clicked (S202: NO), it then determines whether the OK button has been clicked (step S204). The input support application 150 repeatedly executes the processes from step S202 onward until the OK button is clicked.

[0234] When the input support application 150 determines that the OK button has been clicked (S204: YES), it instructs the OS 140 to close the settings screen (step S205). After completing the process in step S205, the input support application 150 terminates the flow. Alternatively, if the OK button is clicked, the application may set the various values ​​before terminating the flow.

[0235] <Effects> The non-contact input device 100 includes a plurality of sensor electrodes 121X and 121Y arranged two-dimensionally along the X-axis (first axis) and Y-axis (second axis) on the back side of the operating surface 105A, an input sensor circuit 125A that measures the two-dimensional position of an object and the distance from the operating surface 105A to the object from the capacitance between each of the plurality of sensor electrodes 121X and 121Y and the fingertip FT (object) that performs non-contact input to the operating surface 105A, a display 110 arranged on top of the plurality of sensor electrodes 121X and 121Y, and a control device 130. The sum of the fluctuation amounts of multiple two-dimensional positions calculated at multiple points in time is calculated as an indicator of the positional accuracy of the fingertip FT (object). The sum of the fluctuation amounts of multiple two-dimensional positions is a value that reflects the variability of the fluctuation amounts of multiple two-dimensional positions in response to external noise and the movement of the fingertip FT.

[0236] Therefore, it is possible to provide a non-contact input device 100 that can calculate positional accuracy in response to external noise and hand movements.

[0237] Furthermore, the amount of variation may be the difference between the moving average of multiple two-dimensional positions at multiple time points and each of the multiple two-dimensional positions at multiple time points. By using the difference between the moving average of multiple two-dimensional positions and each of the two-dimensional positions as the amount of variation, an amount of variation that reflects variability in response to external noise and fingertip FT movement can be obtained, and a non-contact input device 100 that can calculate position accuracy in response to external noise and hand movement with high precision can be provided.

[0238] The non-contact input device 100 displays a pointer at a two-dimensional position, and may also change the size of the pointer according to the magnitude of the fluctuation. By calculating the moving average of the two-dimensional position as the two-dimensional position of the fingertip FT, a two-dimensional position of the fingertip FT that suppresses fluctuations due to external noise and hand movements can be obtained, and the pointer can be displayed on a highly accurate XY coordinate system that suppresses the influence of the fluctuation. Furthermore, since the fluctuation is an indicator of the positional accuracy of the two-dimensional position, the positional accuracy can be visually communicated to the user by changing the size of the pointer according to the fluctuation. In addition, since the fluctuation represents the positional accuracy that also reflects the distance between the fingertip FT and the operating surface 105A, it is possible to provide a non-contact input device 100 that can calculate positional accuracy that reflects multiple physical quantities related to positional accuracy.

[0239] Furthermore, the pointer may be circular or annular, and the non-contact input device 100 may display the pointer so that its center is located at the XY coordinates of the fingertip FT (object). Therefore, it is possible to display a highly accurate, easy-to-see circular or annular pointer centered at the XY coordinates, suppressing fluctuations due to external noise and hand movements, thereby providing a non-contact input device 100 that is easy for the user to operate. Additionally, by indicating the positional accuracy using the pointer's radius, the positional accuracy can be clearly communicated to the user.

[0240] Furthermore, the non-contact input device 100 may switch modes between a first region where the measured capacitance is equal to or greater than a first threshold, a second region where the measured capacitance is less than the first threshold and equal to or greater than a second threshold, and a third region where the measured capacitance is less than the second threshold and equal to or greater than a third threshold. When the measured capacitance becomes equal to or greater than the third threshold, the power to the display 110 is turned off, and the device transitions from standby mode to proximity mode. When the measured capacitance becomes greater than the second threshold, a pointer is displayed. If the state where the measured capacitance is greater than the first threshold continues for a predetermined time, the device may determine the input coordinates resulting from the operation of the object. By displaying a pointer in accordance with the increase in capacitance and determining the operation input when the state where the capacitance is equal to or greater than the largest threshold continues, a non-contact input device 100 that suppresses erroneous inputs can be provided.

[0241] Furthermore, the non-contact input device 100 may calculate the amount of fluctuation in the two-dimensional position per unit time by setting the amount of fluctuation before the measured capacitance exceeds a third threshold (proximity threshold) as a predetermined value. When a finger is suddenly brought close to the operating surface from a distance (from the standby area), a large pointer is displayed first, and then the pointer gradually becomes smaller. This provides a non-contact input device 100 that makes it easy to visually confirm that the positional accuracy is gradually improving.

[0242] Furthermore, the non-contact input device 100 may calculate the amount of variation in the two-dimensional position per unit time using the measured capacitance value after it exceeds the third threshold (proximity threshold). By calculating the position accuracy before displaying the pointer, a pointer of a size appropriate to the position accuracy can be displayed from the moment the pointer is displayed.

[0243] Furthermore, the non-contact input device 100 may determine that the fingertip FT (object) is the palm and that the operation is being performed by the palm if the amount of variation per unit time of multiple two-dimensional positions calculated at multiple points in time is greater than a threshold. When the palm approaches the operating surface 105A and the operation is performed without raising the fingertip FT against the operating surface 105A, the area over which capacitance is detected by the electrostatic sensor 120 increases, and the amount of variation increases. By detecting this state, non-contact operation input by the palm can be detected.

[0244] Although an exemplary embodiment of a contactless input device of this disclosure has been described above, this disclosure is not limited to the specifically disclosed embodiments, and various modifications and changes are possible without departing from the scope of the claims.

[0245] The following additional information is disclosed regarding the embodiments described above. (Note 1) Multiple sensor electrodes are arranged two-dimensionally along the first and second axes on the back side of the operating surface, An input sensor circuit that measures the two-dimensional position of an object and the distance from the operating surface to the object from the capacitance between each of the plurality of sensor electrodes and an object that is subjected to non-contact operation input to the operating surface, A display arranged in conjunction with the aforementioned plurality of sensor electrodes, Control device and Includes, A non-contact input device that calculates the sum of the fluctuation amounts of multiple two-dimensional positions calculated at multiple points in time as an indicator of the positional accuracy of the object. (Note 2) The non-contact input device as described in Appendix 1, wherein the amount of variation is the difference between the moving average of the plurality of two-dimensional positions at the plurality of time points and each of the plurality of two-dimensional positions at the plurality of time points. (Note 3) The aforementioned non-contact input device is A non-contact input device as described in Appendix 2, which displays a pointer at the aforementioned two-dimensional position and changes the size of the pointer according to the magnitude of the fluctuation amount. (Note 4) The pointer is circular or annular, The control device is the non-contact input device according to appended note 3, which displays the pointer so that the center of the pointer is located at the two-dimensional position of the object. (Appended note 5) The non-contact input device switches modes among a first region where the measured capacitance is greater than or equal to a first threshold value, a second region where the measured capacitance is less than the first threshold value and greater than or equal to a second threshold value smaller than the first threshold value, and a third region where the measured capacitance is less than the second threshold value and greater than or equal to a third threshold value smaller than the second threshold value, when the measured capacitance becomes less than or equal to the third threshold value, cuts off the power supply of the display, when the measured capacitance becomes greater than the second threshold value, displays the pointer, The non-contact input device according to appended note 3 or 4, which determines input coordinates by operation of the object when a state where the measured capacitance is greater than the first threshold value continues for a predetermined time. (Appended note 6) The non-contact input device according to appended note 5, which calculates the amount of change per unit time of the two-dimensional position by using, as a predetermined value, the amount of change before the measured capacitance becomes greater than or equal to the third threshold value. (Appended note 7) The non-contact input device according to appended note 5, which calculates the amount of change per unit time of the two-dimensional position by using a value after the measured capacitance becomes greater than or equal to the third threshold value. (Appended note 8)[[ID=2,5]] The non-contact input device according to any one of appended notes 1 to 7, which determines that the object is a palm and an operation is being performed by the palm when the amount of change per unit time of the plurality of two-dimensional positions calculated at the plurality of time points is greater than a threshold value.

Explanation of reference numerals

[0246] 100 Non-contact input device 101 Housing 105 Top panel 105A Operation surface 110 displays 120 electrostatic sensors 121X, 121Y sensor electrodes 125 Wiring board 125A Input Sensor Circuit 125B Image display circuit 130 Control device 131 Control Unit 132 memory 140 OS 150 Input Assistance APIs 160 APP

Claims

1. Multiple sensor electrodes are arranged two-dimensionally along the first and second axes on the back side of the operating surface, An input sensor circuit that measures the two-dimensional position of an object and the distance from the operating surface to the object from the capacitance between each of the plurality of sensor electrodes and an object that is subjected to non-contact operation input to the operating surface, A display arranged in conjunction with the aforementioned plurality of sensor electrodes, Control device and Includes, A non-contact input device that calculates the sum of the fluctuation amounts of multiple two-dimensional positions calculated at multiple points in time as an indicator of the positional accuracy of the object.

2. The non-contact input device according to claim 1, wherein the amount of variation is the difference between the moving average of the plurality of two-dimensional positions at the plurality of time points and each of the plurality of two-dimensional positions at the plurality of time points.

3. The aforementioned non-contact input device is The non-contact input device according to claim 2, wherein a pointer is displayed at the two-dimensional position and the size of the pointer is changed according to the magnitude of the amount of variation.

4. The pointer is circular or ring-shaped, The contactless input device according to claim 3, wherein the control device displays the pointer such that the center of the pointer is located at the two-dimensional position of the object.

5. The aforementioned non-contact input device is The mode is switched between a first region where the measured capacitance is equal to or greater than a first threshold, a second region where the measured capacitance is less than the first threshold and equal to or greater than a second threshold that is less than the first threshold, and a third region where the measured capacitance is less than the second threshold and equal to or greater than a third threshold that is less than the second threshold. When the measured capacitance falls below the third threshold, the power to the display is turned off. When the measured capacitance exceeds the second threshold, the pointer is displayed. The non-contact input device according to claim 3 or 4, wherein if the measured capacitance remains greater than the first threshold for a predetermined period of time, the input coordinates are determined by the manipulation of the object.

6. The non-contact input device according to claim 5, wherein the non-contact input device calculates the amount of variation of the two-dimensional position per unit time, with the amount of variation before the measured capacitance exceeds the third threshold being a predetermined value.

7. The non-contact input device according to claim 6, wherein the non-contact input device calculates the amount of variation of the two-dimensional position per unit time using the value obtained after the measured capacitance becomes equal to or greater than the third threshold.

8. The non-contact input device according to claim 1, wherein if the amount of variation per unit time of the plurality of two-dimensional positions calculated at the plurality of time points is greater than a threshold, the device determines that the object is a palm and that the operation is being performed by the palm.

Images