Non-contact input device and non-contact input method

Abstract

A contactless input device (100) comprises: a three-dimensional operation coordinate detection unit (112) that detects a plurality of three-dimensional operation coordinates by sequentially detecting three-dimensional operation coordinates that are three-axis coordinates of a command input object, which is an object that inputs a command, in a predetermined space; a smoothing processing unit (113) that calculates smoothed three-dimensional operation coordinates by performing smoothing using a first number of three-dimensional operation coordinates among the plurality of three-dimensional operation coordinates, the first number being a predetermined number equal to or larger than two; and an airspace reception range passage determination unit (115) that uses the smoothed three-dimensional operation coordinates to identify passage coordinates that are coordinates through which the command input object has passed through an airspace reception range, the airspace reception range being a range defined in advance in the predetermined space.

Description

[Technical field]

[0001] The present disclosure relates to a non-contact input device and a non-contact input method. [Background technology]

[0002] There exists a device that projects a real image surface including various input areas such as buttons onto a specific area in the air, detects the entry of an operating means such as a finger onto that surface, and performs various operations according to the coordinates where the operating means has entered (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-133284 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional technology, the coordinates detected by the airborne operation means generally contain time-varying noise, and the coordinates vibrate slightly, although the magnitude of the noise varies depending on the detection principle. If this vibration is not eliminated, the accuracy of the detected coordinates will decrease.

[0005] Therefore, one or more aspects of the present disclosure aim to remove noise when an operation is detected in a non-contact manner, thereby enabling the coordinates of the operation to be detected with high accuracy. [Means for solving the problem]

[0006] A non-contact input device according to one aspect of the present disclosure includes a three-dimensional operation coordinate detection unit that detects a plurality of three-dimensional operation coordinates by sequentially detecting three-dimensional operation coordinates, which are coordinates of three axes in a predetermined space, of an instruction input object that is an object for inputting instructions; and a smoothing processing unit that calculates smoothed three-dimensional operation coordinates by performing smoothing using a first number of three-dimensional operation coordinates, the first number being a predetermined number of two or more, included in the plurality of three-dimensional operation coordinates. a low-latency smoothing processing unit that calculates low-latency smoothed three-dimensional operation coordinates by performing smoothing using a second number of three-dimensional operation coordinates, the second number being a predetermined number of two or more, included in the plurality of three-dimensional operation coordinates, with a delay time shorter than the time required for the smoothing processing unit to calculate the smoothed three-dimensional operation coordinates; and using the low-latency smoothed three-dimensional operation coordinates, determining whether the pointing input object has passed through an air acceptance range, which is a predetermined range in the space; Using the smoothed three-dimensional operation coordinates, Record The present invention is characterized by comprising a passage determination unit that specifies passing coordinates that are coordinates through which the pointing input object passes through an intermediate reception range.

[0007] A non-contact input method according to one aspect of the present disclosure includes: detecting a plurality of three-dimensional operation coordinates by sequentially detecting three-dimensional operation coordinates, which are coordinates of three axes in a predetermined space, of an instruction input object that is an object for inputting an instruction; calculating smoothed three-dimensional operation coordinates by smoothing a first number of three-dimensional operation coordinates, the first number being a predetermined number equal to or greater than two, included in the plurality of three-dimensional operation coordinates; calculating low-latency smoothed three-dimensional operation coordinates by smoothing using a second number of three-dimensional operation coordinates, the second number being a predetermined number equal to or greater than two, included in the plurality of three-dimensional operation coordinates, with a delay time shorter than a time required for calculating the smoothed three-dimensional operation coordinates; and determining whether the pointing input object has passed through an air acceptance range, which is a predetermined range in the space, using the low-latency smoothed three-dimensional operation coordinates; Using the smoothed three-dimensional operation coordinates, Record The method further comprises identifying passing coordinates, which are coordinates through which the pointing input object passes through an intermediate reception range. Effect of the Invention

[0008] According to one or more aspects of the present disclosure, when inputting an operation in a non-contact manner, noise generated when an operating tool is detected can be removed, and the coordinates of the operating tool can be detected with high accuracy. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a perspective view showing the appearance of a non-contact input device according to first to fourth embodiments. [Diagram 2] FIG. 1 is a block diagram illustrating a schematic configuration of a non-contact input device according to first to third embodiments. [Diagram 3] 13(A) to 13(D) are schematic diagrams for explaining a method for detecting three-dimensional operation coordinates. [Figure 4]FIG. 2 is a schematic diagram for explaining an imaging device coordinate system. [Diagram 5] 10 is a schematic diagram for explaining the correspondence relationship between an imaging device coordinate system and an aerial reception range coordinate system. FIG. [Figure 6] 1A and 1B are block diagrams illustrating an example of a hardware configuration. [Figure 7] 10 is a flowchart showing an operation of determining whether or not a pointing input object has passed through one air reception range in the first embodiment. [Figure 8] 13(A) to 13(C) are graphs for explaining in detail the difference between passing judgment and coordinate judgment. [Figure 9] 13(A) to 13(C) are graphs for explaining values ​​for performing passage judgment and coordinate judgment. [Figure 10] FIG. 11 is a perspective view for explaining the airborne reception range. [Figure 11] 13 is a flowchart showing an operation for identifying a process based on an input to an airborne reception range. [Figure 12] 1 is a graph for explaining a situation in which chattering occurs. [Figure 13] 1 is a graph for explaining a method for eliminating chattering. [Figure 14] FIG. 11 is a schematic diagram showing a projected image in the second embodiment. [Figure 15] 13 is a diagram showing the vicinity of the aerial reception range in the second embodiment cut by a plane perpendicular to the y-axis. FIG. [Figure 16] 13 is a flowchart showing an operation of determining whether a pointing input object has passed through one air reception range in the second embodiment. [Figure 17] FIG. 11 is a schematic diagram showing a modified example of the air reception range in the second embodiment. [Figure 18] 13(A) to 13(C) are graphs for explaining in detail the operation of the aerial reception range passage determination unit in the third embodiment. [Figure 19] 1A and 1B are schematic diagrams showing the configuration of a two-stage cascade type IIR filter. [Figure 20]1 is a graph showing group delay characteristics of a two-stage cascade type IIR filter. [Figure 21] FIG. 13 is a block diagram illustrating a schematic configuration of a non-contact input device according to a fourth embodiment. [Figure 22] 13 is a flowchart showing the operation of determining whether a pointing input object has passed through one air reception range in the fourth embodiment. [Diagram 23] 13A to 13C are graphs for explaining the operation of the aerial reception range passage determination unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Embodiment 1 FIG. 1 is a perspective view showing the appearance of a non-contact input device 100 according to the first embodiment. Non-contact input device 100 projects aerial image 102 into a predetermined space by a mechanism mounted inside housing 101. A known technique may be used for the mechanism for projecting aerial image 102, and detailed description thereof will be omitted. As a known technique, for example, a display is mounted inside housing 101, and a retroreflective material and a half mirror are used to re-image light emitted from the display in space, thereby projecting aerial image 102. Note that another means may be used for the mechanism for this projection.

[0011] Buttons 103a, 103b, and 103c, which are an aerial input receiving surface, are displayed in aerial image 102. Each of buttons 103a, 103b, and 103c is assigned an instruction to non-contact input device 100. Here, the instruction "OK" is assigned to button 103a, the instruction "Cancel" is assigned to button 103b, and the instruction "Set" is assigned to button 103c. When it is not necessary to distinguish between the buttons 103a, 103b, and 103c, one of the buttons 103a, 103b, and 103c will be referred to as the button 103.

[0012] Further, each of the buttons 103a, 103b, and 103c is associated with an air reception range that is a range for receiving an instruction input from a user. For example, when an instruction input object such as a finger enters the air reception range associated with the button 103a, the non-contact input device 100 determines that an instruction of the content assigned to the button 103a has been input. The instruction input object is an object that inputs an instruction and is used to input an instruction. For example, the instruction input object is a finger or a protruding object such as a stick.

[0013] The aerial reception range may be a part of the aerial image 102. The aerial reception range may not coincide with the button 103 shown in the aerial image 102, for example, it may be offset from the associated button 103, inclined from the associated button 103, curved with respect to the associated button 103, etc. Furthermore, the aerial reception range may be a three-dimensional shape with thickness, such as a rectangular parallelepiped or a sphere, in addition to a flat or curved surface.

[0014] A user of the non-contact input device 100 gives instructions to the non-contact input device 100 by inserting a pointing input object such as a finger into an air reception range associated with the button 103 of the desired instruction. Detection of whether or not the pointing input object has entered the air reception range is performed using an image capturer 104 that is attached to the housing 101 of the non-contact input device 100 and has an imaging range that includes the air reception ranges associated with the buttons 103a, 103b, and 103c.

[0015] The image pickup device 104 is a detection device capable of capturing a three-dimensional image. The image pickup device 104 is, for example, a three-dimensional image pickup device using a TOF (Time Of Flight) method. However, the three-dimensional image pickup device using the TOF method is merely an example, and the image pickup device 104 may be any device that is a detection device capable of acquiring three-dimensional information, such as a twin-lens stereo camera, an active stereo camera, or a LiDAR.

[0016] When an instruction is given, the non-contact input device 100 outputs an operation acceptance sound from the speaker 105 as feedback to the user. In the first embodiment, the feedback is given by sound, but this is only one example, and the feedback may be given by, for example, a pseudo-tactile sensation given by emitting light, vibration, or ultrasonic waves to a finger, or a combination of these.

[0017] FIG. 2 is a block diagram showing a schematic configuration of the non-contact input device 100 according to the first embodiment. Non-contact input device 100 includes a three-dimensional coordinate acquisition unit 110 , a smoothing processing unit 113 , an aerial image projection unit 114 , an aerial reception range passage determination unit 115 , a feedback unit 116 , and an output unit 117 .

[0018] The three-dimensional coordinate acquisition unit 110 acquires three-dimensional operation coordinates which are the three-dimensional coordinates of a pointing input object. The three-dimensional coordinate acquisition unit 110 includes a three-dimensional image capturing unit 111 and a three-dimensional operation coordinate detection unit 112 .

[0019] The three-dimensional image capturing section 111 is a functional section realized by the image capturing device 104 in Fig. 1. The three-dimensional image capturing section 111 captures a three-dimensional image IM, and provides the three-dimensional image IM to a three-dimensional operation coordinate detection section 112. Specifically, the three-dimensional image capturing section 111 outputs a three-dimensional image IM, which is a three-dimensional image of a predetermined space, so as to include an aerial image projected by an aerial image projection section 114, which will be described later.

[0020] The three-dimensional operation coordinate detection unit 112 detects three-dimensional operation coordinates C, which are the coordinates of the three axes of the pointing input object, from the three-dimensional image IM. Here, the three-dimensional operation coordinate detection unit 112 detects three-dimensional operation coordinates C of the pointing input object in sequence, thereby detecting a plurality of three-dimensional operation coordinates C at different times.

[0021] 3(A) to (D) are schematic diagrams for explaining a method for detecting the three-dimensional operation coordinate C. FIG. 3A shows a three-dimensional image IM. The three-dimensional image IM here is represented by a horizontal axis X and a vertical axis Y, and is a distance image in which the shade of a pixel indicates the distance from the image pickup device 104. Here, in the three-dimensional image IM, the greater the brightness of a pixel, the closer the pixel is to the image pickup device 104.

[0022] FIG. 3(B) shows a binarized three-dimensional image IM#1 generated by binarizing the three-dimensional image IM shown in FIG. 3(A). The three-dimensional operation coordinate detection unit 112 generates a binarized three-dimensional image IM#1 by binarizing the three-dimensional image IM. The binarization is performed to distinguish between points of a pointing input object such as a finger and other points. Therefore, it is assumed that a threshold value is determined so that such a distinction can be made appropriately. For example, as shown in FIG. 1, when the image capture device 104 is placed facing downward, it is desirable to set the threshold value to a value farther than the farthest end of the air reception range corresponding to each of the buttons 103a, 103b, and 103c and closer to the ground.

[0023] Next, the three-dimensional operation coordinate detection unit 112 performs noise removal on the binarized three-dimensional image IM#1 to generate a noise-removed three-dimensional image IM#2. FIG. 3C is a schematic diagram showing the noise-removed three-dimensional image IM#2. Distance images generally have relatively large noise. For this reason, the binarized three-dimensional image IM#1 shown in FIG. 3B also has a lot of fine noise. In response to this, the three-dimensional operation coordinate detection unit 112 removes noise by increasing the minimum area to be detected as a closed region to a certain extent, as in the noise-removed three-dimensional image IM#2 shown in FIG. 3C. Note that various other methods of removing noise can be applied, such as a method using a median filter or a method of performing opening after closing.

[0024] Next, the three-dimensional operation coordinate detection unit 112 detects the tip of the finger, which is the pointing input object, from the noise-removed three-dimensional image IM#2, as shown in Fig. 3(D). Here, the negative side of the Y axis is the user side of the air surface, and the positive side of the Y axis is the housing 101 side. Therefore, since the finger enters from the negative side to the positive side of the Y axis, the tip of the finger is simply the point with the smallest Y axis coordinate among the detected closed regions. In Fig. 3(D), this point is indicated by a cross symbol.

[0025] The X and Y coordinates of the tip point are determined by the above. Furthermore, the distance of this point is the depth Z obtained from the brightness of the (X, Y) pixel in the original 3D image IM, so the 3D coordinates (X, Y, Z) are determined.

[0026] Next, the three-dimensional operation coordinate detection unit 112 expresses the three-dimensional coordinates (X, Y, Z) in an image capture coordinate system, which is a coordinate system with the three-dimensional image capture unit 111 as the origin. As shown in FIG. 4, the image capture coordinate system has the tip of the optical axis of the lens 104a of the image capture unit 104 used as the three-dimensional image capture unit 111 as the origin, and the long axis side is x c axis, the short axis side is y c axis, the optical axis direction is z c In a typical non-telecentric optical system, the imaging area spreads radially, so if the Z coordinate is different for the same X or Y coordinate, the x c Or y c The values ​​of will be different.

[0027] As shown in Figure 4, the horizontal angle of view is α, the vertical angle of view is β, and the range of X is (-X max ,X max ), and the range of Y is (-Y max ,Y max ), the three-dimensional coordinates (X, Y, Z) of the fingertip in the captured image are converted into the three-dimensional captured coordinates (x c ,y c ,z c ) can be converted to the following equation (1):

number

[0028] The above-described method of conversion into three-dimensional image coordinates is one example, and various other methods such as a method using pattern matching or a learning device can be applied.

[0029] The three-dimensional coordinates (x c ,y c ,z c ) requires further transformation if the camera coordinate system and the coordinate system of the aerial reception range are different. This transformation is done by using the imaged 3D coordinates (x c ,y c ,z c ) into a three-dimensional operation coordinate C expressed in an airborne reception range coordinate system having x-, y-, and z-axes.

[0030] FIG. 5 is a schematic diagram for explaining the correspondence relationship between the imaging device coordinate system and the aerial reception range coordinate system. This conversion is performed as many times as there are aerial reception ranges R, but for simplicity's sake, FIG. 5 shows only one aerial reception range R1.

[0031] In FIG. 5, as a specific example, the air reception range R1 associated with a certain button 103 is (|x| <x max ,|y| <y max The origin of the aerial reception range coordinate system is (x c0 ,y c0 ,z c0 ), x c Rotation around the axis is θ, y c The rotation around the axis is ψ, z c If the rotation around the axis is φ, the following equation (2) is used to calculate the three-dimensional image coordinate (x c ,y c ,z c ) can be transformed into (x, y, z), which is the three-dimensional operation coordinate C in the airborne reception range coordinate system.

number

[0032] As mentioned above, the conversion from the camera coordinate system to the aerial reception range coordinate system needs to be performed for each aerial reception range R. However, by making the normal direction of the aerial reception range R common, it is possible to standardize costly rotation calculations and reduce the amount of calculations.

[0033] Returning to Figure 2, the smoothing processing unit 113 calculates smoothed three-dimensional operation coordinates SC by performing smoothing using a first number of three-dimensional operation coordinates C, which is a predetermined number of two or more, included in the multiple three-dimensional operation coordinates C. For example, the smoothing processing unit 113 smoothes the three-dimensional operation coordinate C in the time direction, and provides the result as the smoothed three-dimensional operation coordinate SC to the aerial reception range passage determination unit 115. The smoothing method in this embodiment is smoothing by a seven-tap moving average. With (x, y, z) at time t as x(t), y(t) and z(t), the smoothing processing unit 113 calculates the smoothed three-dimensional operation coordinate SC by the following formula (3).

number

[0034] In this embodiment, a moving average filter is used as the smoothing means, but this embodiment is not limited to this example. As the smoothing means, various means can be applied, such as a means using an FIR (Finite Impulse Response) digital filter or an IIR (Infinite Impulse Response) digital filter with different coefficients such as a window function method as a smoothing filter.

[0035] The aerial image projection unit 114 projects an aerial image having a portion associated with the aerial reception range into a predetermined space. For example, the aerial image projection unit 114 projects an aerial image 102 as shown in Fig. 1. In the aerial image 102, the portion associated with the aerial reception range is buttons 130a to 130c. A known technique may be used to project the aerial image. Here, aerial image projection section 114 provides aerial reception range information indicating aerial reception range R in the aerial reception range coordinate system to aerial reception range pass determination section 115.

[0036] Aerial acceptance range pass determination unit 115 is a pass determination unit that determines whether or not a finger, which is an example of an instruction input object, has passed through aerial acceptance range R, based on three-dimensional operation coordinates C and aerial acceptance range R indicated by aerial acceptance range information from aerial image projection unit 114. Then, aerial acceptance range pass determination unit 115 provides a pass determination result P indicating the determination result to feedback unit 116.

[0037] Here, the aerial acceptance range passage determination unit 115 determines whether or not the pointing input object has passed through the aerial acceptance range R by using a predetermined function such that the output of the function, which has as input the three-dimensional operation coordinate C, becomes a predetermined value in the aerial acceptance range R. For example, the aerial acceptance range passage determination unit 115 can sequentially input a plurality of three-dimensional operation coordinates to the function, and determine whether or not the pointing input object has passed through the aerial acceptance range R when the output of the function becomes larger or smaller than that value.

[0038] Specifically, the aerial acceptance range passage determination unit 115 sets a function f(x, y, z) such that a point on the aerial acceptance range R satisfies f(x, y, z) = 0. As a result, when f(x, y, z) changes from positive to negative or from negative to positive, it always passes through the aerial acceptance range R set as f(x, y, z) = 0. Therefore, the aerial acceptance range passage determination unit 115 can determine whether or not the pointing input object has passed through the aerial acceptance range by checking the value of f(x, y, z) inputted with the three-dimensional operation coordinates C, in other words, by using the three-dimensional operation coordinates C. In the first embodiment, the description will be given assuming that the aerial reception range R is a plane where f(x, y, z)=z.

[0039] Furthermore, the air acceptance range passage determination unit 115 identifies passage coordinates, which are the coordinates at which the pointing input object passes through the air acceptance range R, using the smoothed three-dimensional operation coordinates SC. Specifically, the aerial acceptance range pass determination unit 115 identifies pass coordinates CSC indicating which coordinates in the aerial acceptance range R the finger, which is the pointing input object, has passed through, based on the smoothed three-dimensional operation coordinates SC from the smoothing processing unit 113.

[0040] Here, the passing coordinate CSC is composed of the x-axis value and the y-axis value calculated by the above equation (3) at the time t when it is determined that a finger, which is an example of an instruction input object, has passed through the air reception range R from the three-dimensional operation coordinate C.

[0041] Furthermore, when the identified passing coordinate CSC is within the aerial reception range R, the aerial reception range pass determination unit 115 determines that the pointing input object has passed through the aerial reception range R, in other words, that input has been made into the aerial reception range R.

[0042] For example, the aerial acceptance range passing determination unit 115 specifies the value of a first axis extending in a direction intersecting the aerial acceptance range R from the three axes from the three-dimensional operation coordinates C. The aerial acceptance range passing determination unit 115 specifies the time when the instruction input object passed through the aerial acceptance range R using the specified value. The aerial acceptance range passing determination unit 115 specifies the values ​​of the second axis and the third axis, excluding the first axis from the three axes, at the specified time as the passing coordinates CSC from the smoothed three-dimensional operation coordinates SC. Then, when the passing coordinates CSC are within the aerial acceptance range R, the aerial acceptance range passing determination unit 115 determines that an input has been made to the aerial acceptance range R. Here, the first axis is the z-axis, and the second axis and the third axis are the x-axis and the y-axis.

[0043] The feedback unit 116 refers to the passage determination result P, and performs feedback when the determination is true, in other words, when the determination indicates that the instruction input object has passed through the air reception range R. Specifically, the feedback unit 116 causes the output unit 117 serving as the speaker 105 to output an operation reception sound, for example.

[0044] The output unit 117 performs output to the outside of the non-contact input device 100 in response to an instruction from the feedback unit 116. The feedback unit 116 performs an instruction when the instruction input object passes through the air reception range R. Here, the output unit 117 outputs feedback to the user. For example, when the output unit 117 is the speaker 105 shown in Fig. 1, the output unit 117 outputs an operation acceptance sound. The output unit 117 is not limited to the speaker 105 that outputs sound, and may be other means such as a display. The output unit 117 may also output ultrasonic waves that give a tactile sensation to the pointing input object.

[0045] The three-dimensional operation coordinate detection unit 112, the smoothing processing unit 113, the aerial reception range passage determination unit 115, and the feedback unit 116 described above may be partly or entirely configured with a memory 10 and a processor 11 such as a CPU (Central Processing Unit) that executes a program stored in the memory 10, as shown in FIG. 6(A). Such a program may be provided through a network, or may be provided by being recorded on a recording medium. That is, such a program may be provided, for example, as a program product.

[0046] In addition, some or all of the three-dimensional operation coordinate detection unit 112, the smoothing processing unit 113, the aerial reception range passage judgment unit 115 and the feedback unit 116 can also be configured as a processing circuit 12 such as a single circuit, a composite circuit, a processor operated by a program, a parallel processor operated by a program, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), for example, as shown in Figure 6 (B). As described above, the three-dimensional operation coordinate detection unit 112, the smoothing processing unit 113, the aerial reception range passage determination unit 115, and the feedback unit 116 can be realized by a processing circuit network.

[0047] FIG. 7 is a flowchart showing the operation of determining whether a pointing input object has passed through one air reception range R in the first embodiment. Here, the air reception range R is defined as the air reception range R1(|x| <x max ,|y| <y max , z=0).

[0048] First, the aerial reception range passage determination unit 115 determines whether z(t-1), which is the z coordinate value at time t-1 in the three-dimensional operation coordinate C, is positive or 0, and whether z(t), which is the z coordinate value at time t in the three-dimensional operation coordinate C, is negative (S10). When the frame rate is 30 frames per second, the time difference between time t-1 and time t is 1 / 30 seconds, which is the time difference for one frame.

[0049] This is because, in the aerial reception range coordinate system, the origin is set on the aerial reception range R1, the upward direction of the vertical line of the aerial reception range R1 is positive, and instructions are input by the object passing through the aerial reception range R1 from top to bottom, as shown in Figure 5. Therefore, the aerial reception range R1 is a plane where z=0.

[0050] This corresponds to the so-called falling edge of z(t). It is important to note here that the value used for the judgment is the three-dimensional operation coordinate C, which is the value before smoothing. This is because the smoothed three-dimensional operation coordinate SC has a delay in exchange for reducing error, and so if the smoothed three-dimensional operation coordinate SC is used, the judgment will also be delayed. On the other hand, since immediate response is important for the performance of the device, the three-dimensional operation coordinate C is used here.

[0051] Next, the aerial reception range passage judgment unit 115 judges whether the passage coordinate CSC, which is the smoothed value of the x coordinate and the smoothed value of the y coordinate at time t in the smoothed three-dimensional operation coordinate SC, satisfies the following equations (4) and (5) (S11).

number

number

[0052] If both equations (4) and (5) are satisfied (Yes in S11), the process proceeds to step S12; if at least one of equations (4) and (5) is not satisfied (No in S11), the process proceeds to step S13.

[0053] In step S12, the air acceptance range passage determination unit 115 determines that the pointing input object has passed through one of the air acceptance ranges R that is a target. On the other hand, in step S13, the air acceptance range passage determination unit 115 determines that the pointing input object has not passed through one of the target air acceptance ranges R.

[0054] In addition, if multiple aerial reception ranges R are provided, the above flowchart can be performed for all aerial reception ranges R, allowing the aerial reception range pass-through determination unit 115 to identify which aerial reception range R the pointing input object has passed through.

[0055] Here, the judgment in step S11 is to judge whether the smoothed three-dimensional operation coordinate SC is within the air reception range R or not. This is, for example, a judgment to specify which button 103 on the screen has been pressed, and in order to increase the substantial resolution, it is necessary to use the smoothed three-dimensional operation coordinate SC, which is a highly accurate coordinate. As a trade-off, a delay occurs, but in reality, when a user of the non-contact input device 100 such as this embodiment operates the button 103 in the air, it is known that the user tends to move the finger in a direction perpendicular to the button 103, and when moving the finger in this direction, the x coordinate and the y coordinate are almost constant, so a slight delay does not pose a problem.

[0056] The reason that different smoothing strategies are effective for passage judgment and coordinate judgment in this way is that passage judgment is a binary judgment of whether or not there is passage, while coordinate judgment is a multi-valued judgment in that there are as many variations in the judgment result as there are coordinates on the screen.

[0057] 8(A) to 8(C) are graphs for explaining in detail the difference between the passage judgment and the coordinate judgment as described above.

[0058] Fig. 8(A) is a graph showing a time series of the x-axis value of the actual finger movement in the air reception range coordinate system, the x-axis value of the three-dimensional operation coordinate C, and the x-axis value of the smoothed three-dimensional operation coordinate SC. The horizontal axis of Fig. 8(A) indicates time, and the vertical axis indicates the x-axis value.

[0059] Fig. 8(B) is a graph showing a time series of the y-axis value of the actual finger movement in the air reception range coordinate system, the y-axis value of the three-dimensional operation coordinate C, and the y-axis value of the smoothed three-dimensional operation coordinate SC. The horizontal axis of Fig. 8(B) indicates time, and the vertical axis indicates the y-axis value.

[0060] Fig. 8(C) is a graph showing a time series of the z-axis value of the actual finger movement in the air reception range coordinate system, the z-axis value of the three-dimensional operation coordinate C, and the z-axis value of the smoothed three-dimensional operation coordinate SC. The horizontal axis of Fig. 8(C) indicates time, and the vertical axis indicates the z-axis value.

[0061] In the graphs shown in Figures 8(A) to (C), the solid lines represent true values ​​as actual finger coordinates, the dashed lines represent measured values ​​of three-dimensional operation coordinates C, and the double lines represent smoothed values ​​of three-dimensional operation coordinates SC obtained by smoothing the values ​​of three-dimensional operation coordinates C.

[0062] 8(A) to (C), the value of the three-dimensional operation coordinate C shown by the dashed line has an error and oscillates compared to the true value shown by the solid line. The value of the smoothed three-dimensional operation coordinate SC shown by the double line has an error mitigated compared to the value of the three-dimensional operation coordinate C shown by the dashed line, but there is a delay of about three frames compared to the true value shown by the solid line.

[0063] Here, the condition of step S10 shown in FIG. 7 is satisfied (Yes in S10) when the value of the z-axis in the three-dimensional operation coordinate C falls at t=15, as shown by the dashed line in FIG. 8(C). Then, in step S11, a determination is made using the x-axis value and the y-axis value of the smoothed three-dimensional operation coordinate SC at t=15.

[0064] The above relationship is shown in the graphs of FIGS. 9(A) to (C). Furthermore, the value of the smoothed three-dimensional operation coordinate SC is also applied to the passing coordinate CSC, which indicates which coordinate on this button 103 has been passed through.

[0065] The passing coordinate CSC can be used, for example, to select a cell when the aerial screen corresponding to the aerial acceptance range R is a worksheet with many cells. Even when the area where a drawing is performed, such as the signature section of a credit card, is the aerial acceptance range R, the signature can be identified by the trajectory of the passing coordinate CSC. Furthermore, even when there are several buttons 103 on the screen, as shown in Fig. 10, the entire screen can be covered by one aerial acceptance range R, and it can be determined which button 103 has been pressed.

[0066] As can be seen from the graph above, there is a delay of about three frames in these smoothed values. However, as mentioned above, when the z-axis value in three-dimensional operation coordinate C falls, the x-axis value and y-axis value in three-dimensional operation coordinate C do not fluctuate much, so this is not a problem in practical terms.

[0067] In FIG. 9, when the z-axis value in the three-dimensional operation coordinate C falls, the x-axis value and the y-axis value in the three-dimensional operation coordinate C fluctuate. However, this is merely a value that has been intentionally manipulated to easily express the delay, and in any case, this is not a problem.

[0068] In this way, the aerial reception range pass determination unit 115 determines whether or not the aerial reception range R has been passed through as a pass determination result P. The aerial reception range pass determination unit 115 also determines pass coordinates CSC indicating which coordinates within the aerial reception range R have been passed through.

[0069] In the first embodiment, the passing coordinate CSC is the x-axis value and the y-axis value at the moment t=15 when the falling edge is observed, but the first embodiment is not limited to this example. The passing coordinate CSC may be a weighted average with the previous time t=14, for example, as in the following formula (6).

number

[0070] Furthermore, the passing coordinate CSC may be a value obtained by further adding weighting based on the z coordinate at t=15, as in the following formula (7).

number

[0071] Although a finger is used as the pointing input object here, other means such as a stick may be used. Furthermore, when it is not easy to distinguish between a finger as a pointing input object and other objects, or when such a distinction is not necessary, the three-dimensional coordinate acquisition unit 110 and the air reception range passage determination unit 115 may determine that an operation has been accepted if any object has entered. This means that any protruding object is allowed as a pointing input object.

[0072] When the entire screen is covered by a single aerial reception range R, it is possible to determine which process is to be performed according to the flow shown in FIG.

[0073] FIG. 11 is a flowchart showing the operation of specifying a process based on an input to the airborne reception range R.

[0074] First, the air acceptance range passage determination unit 115 determines whether or not the pointing input object has passed through the air acceptance range R (S20) in the same manner as in steps S10 and S11 of Fig. 7. If the pointing input object has passed through the air acceptance range R, the process proceeds to step S21.

[0075] In step S21, the aerial reception range passage determination unit 115 determines whether or not the passage coordinate CSC is within the range assigned to the "Decision" button 103a, for example. If the passage coordinate CSC is within the range assigned to the "Decision" button 103a (Yes in S21), the process proceeds to step S22, and if the passage coordinate CSC is not within the range assigned to the "Decision" button 103a (No in S21), the process proceeds to step S23.

[0076] In step S22, the aerial reception range passage determination unit 115 executes the process assigned to "Decision".

[0077] In step S23, the aerial reception range passage determination unit 115 determines whether the passage coordinate CSC is within the range assigned to the "Cancel" button 103b, for example. If the passage coordinate CSC is within the range assigned to the "Cancel" button 103b (Yes in S23), the process proceeds to step S24, and if the passage coordinate CSC is not within the range assigned to the "Cancel" button 103b (No in S23), the process proceeds to step S25.

[0078] In step S24, the aerial reception range passage determination unit 115 cancels the process.

[0079] In step S25, the aerial reception range passage determination unit 115 determines whether the passage coordinate CSC is within the range assigned to the "Set" button 103c, for example. If the passage coordinate CSC is within the range assigned to the "Set" button 103c (Yes in S25), the process proceeds to step S26, and if the passage coordinate CSC is not within the range assigned to the "Set" button 103c (No in S25), the process ends.

[0080] In step S26, the aerial reception range passage determination unit 115 executes the process assigned to "Setting".

[0081] In the above-described first embodiment, the moment when button 103 is pressed, in other words, the moment when the z-axis value in the three-dimensional operation coordinate C falls, is regarded as the moment of operation, but the first embodiment is not limited to this example. For example, the moment when the z-axis value in the three-dimensional operation coordinate C rises may be determined as the moment of operation.

[0082] In addition, in the first embodiment, fingertip detection is performed first, and then coordinate conversion is performed from the camera coordinate system to the airborne reception range coordinate system, but the first embodiment is not limited to such an example. For example, the coordinates of the airborne reception range R may be reversely converted from the airborne reception range coordinate system to the camera coordinate system in advance. In this case, it is sufficient to determine the passage of the airborne reception range R and the coordinates at the time of passing through it in the camera coordinate system.

[0083] Furthermore, when multiple operations are permitted, so-called chattering occurs, in which multiple operations are determined to have been performed even when the user intended to perform only one operation, due to vibrations caused by an error in the z-axis value in the three-dimensional operation coordinate C. For example, when the z-axis value in the three-dimensional operation coordinate C transitions as shown in the graph in FIG. 12, in addition to the time t=15, which is the timing when the finger actually passed, it is erroneously determined that the finger passed at t=17. In order to prevent such a situation, it is effective to change the threshold value of the falling edge of the z-axis value in the three-dimensional operation coordinate C, in other words, the threshold value from the positive side to the negative side, and the threshold value of the rising edge of the z-axis value in the three-dimensional operation coordinate C, in other words, the threshold value from the negative side to the positive side. Specifically, it is sufficient to make the threshold value for detecting the passage of the pointing input object through the air reception range R different from the threshold value for detecting the return of the pointing input object to the side before passing through the plane after the pointing input object has passed through the air reception range R.

[0084] This is specifically shown in the graph in Figure 13. Here, the transition of z(t), which is the z-axis value in the three-dimensional operation coordinate C, is the same as that in the graph shown in Figure 12, but the falling threshold is set to "0" and the rising threshold is set to "1." As a result, the threshold is "0" until time t=15, and after time t=15 when a falling edge is observed, the threshold changes to "1." This eliminates the chattering that existed at time t=17 in Figure 12. This threshold "1" returns to "0" again at time t=25 when a rising edge is observed.

[0085] Embodiment 2 As shown in FIG. 1, non-contact input device 200 according to embodiment 2, like non-contact input device 100 according to embodiment 1, projects an aerial image 102 into the air by a mechanism mounted inside housing 101. Non-contact input device 200 detects operations on buttons 103a, 103b, and 103c included in aerial image 102 using image capture device 104, and notifies through speaker 105 that the operations have been performed.

[0086] As shown in FIG. 2, the non-contact input device 200 in the second embodiment includes a three-dimensional coordinate acquisition unit 110, a smoothing processing unit 113, an aerial image projection unit 214, an aerial reception range pass-through determination unit 215, a feedback unit 116, and an output unit 117.

[0087] The three-dimensional coordinate acquisition unit 110, the smoothing processing unit 113, the feedback unit 116 and the output unit 117 of the non-contact input device 200 in embodiment 2 are similar to the three-dimensional coordinate acquisition unit 110, the smoothing processing unit 113, the feedback unit 116 and the output unit 117 of the non-contact input device 100 in embodiment 1. However, in embodiment 2, the aerial image projection unit 214 provides the aerial reception range R to the aerial reception range pass judgment unit 215, which receives three-dimensional operation coordinates C from the three-dimensional operation coordinate detection unit 112 and receives smoothed three-dimensional operation coordinates SC from the smoothing processing unit 113, provides the pass judgment result P to the feedback unit 116, and outputs the pass coordinates CSC to the output unit 117.

[0088] Aerial image projection unit 214 projects into the air a curved surface that is a part of a sphere as shown in Fig. 14. The aerial reception range R in the second embodiment is assumed to coincide with the curved surface. Note that the aerial reception range R shown in Fig. 14 is displayed with a mesh to make it easier to visually grasp the shape.

[0089] The aerial reception range R, which is a curved surface that is a part of a sphere, has some advantages as a user interface over a flat surface. For example, it is considered that a user generally operates the non-contact input device 200 near the center of the aerial reception range R. However, if the aerial reception range R is a flat surface as in the first embodiment, the distance from the user to the aerial reception range R will be significantly different between the center of the aerial reception range R and its edge. On the other hand, if the aerial reception range R is a curved surface, the difference in distance can be eliminated or reduced. However, the aerial reception range R shown in FIG. 14 is exaggerated to emphasize the difference from the first embodiment, and in practice, an aerial reception range R with a slightly higher curvature, in other words, a flat aerial reception range R, is more useful.

[0090] The aerial acceptance range passage determination unit 215 expresses the aerial acceptance range R as a function f(x, y, z)=0 indicating a surface including the aerial acceptance range R, and further expresses the aerial acceptance range R as a function g x (x,y,z) and g y When expressed as (x,y,z), g x (x,y,z) and g y The aerial reception range R is limited by the constraints (x, y, z).

[0091] First, since the air reception range R in the second embodiment is a part of a sphere, f(x, y, z) is expressed by a function indicating the sphere. r, y r, zr), and the radius is d, then f(x,y,z)=(xx r ) 2 +(yy r ) 2 +(zz r ) 2 -d 2 It is.

[0092] Also, g x Let (x, y, z) be the length of the arc AB shown in Figure 15. Note that Figure 15 is a diagram of the vicinity of the aerial reception range R cut by a plane perpendicular to the y-axis. Here, from the arc length equation, the length of the arc AB is dθ. However, since the hypotenuse of θ is d and the opposite side is x-xr, the following equation (8) holds.

number

[0093] In other words, the following equation (9) holds:

number

[0094] Similarly, g y Also, the following equation (10) holds.

number

[0095] Here, it is assumed that the coordinates (x,y,z) are in the air reception range R. More precisely, g x and g y By replacing (x, y, z) at the intersection of the line connecting (x, y, z) and the center of the sphere with f(x), the following equations (11) and (12) hold.

number

number

[0096] However, in reality, x and g y As will be described later, the reason why is used is the moment when the pointing input object crosses f(x, y, z), and the coordinates (x, y, z) are sufficiently close to a spherical surface, so there is no significant difference even if equations (9) and (10) are used instead of equations (11) and (12).

[0097] As described above, the coordinates in the air reception range R are determined, and the limit of the coordinate area is |g x (x,y,z)| <x max , |g y (x,y,z)| <y max , z>0, it is possible to represent an aerial reception range R which is a part of a sphere as shown in FIG.

[0098] To simplify further, the inverse sine function is sin in the small angle range. -1 Since we can approximate x = x, we can further simplify equations (9) and (10) to g x (x,y,z)=xx r , g y (x,y,z)=yy r It is also possible to use the following.

[0099] Based on these equations, aerial acceptance range pass determination unit 215 in the second embodiment determines whether or not a finger, which is an example of a pointing input object, has passed through aerial acceptance range R, from three-dimensional operation coordinates C and aerial acceptance range R indicated by aerial acceptance range information from aerial image projection unit 114. Then, aerial acceptance range pass determination unit 215 provides feedback unit 116 with a pass determination result P indicating the determination result.

[0100] Here, the aerial acceptance range passage determination unit 215 sets a function f(x, y, z) such that a point on the aerial acceptance range R satisfies f(x, y, z) = 0. As a result, when f(x, y, z) changes from positive to negative or from negative to positive, it always passes through the aerial acceptance range R set as f(x, y, z) = 0. Therefore, the aerial acceptance range passage determination unit 215 can determine whether or not the pointing input object has passed through the aerial acceptance range by checking the value of f(x, y, z). In the second embodiment, the air reception range R is a portion of a sphere, as shown in FIG.

[0101] Furthermore, the aerial acceptance range passage determination unit 215 identifies passage coordinates, which are the coordinates at which the pointing input object passes through the aerial acceptance range R, using the smoothed three-dimensional operation coordinates SC. Specifically, the aerial acceptance range pass determination unit 215 identifies pass coordinates CSC indicating which coordinates in the aerial acceptance range R the finger, which is the pointing input object, has passed through, based on the smoothed three-dimensional operation coordinates SC from the smoothing processing unit 113.

[0102] Here, the passing coordinate CSC is composed of the x-axis value, y-axis value, and z-axis value in the smoothed three-dimensional operation coordinate SC at time t when it is determined that a finger, which is an example of an instruction input object, has passed through the air reception range R from the three-dimensional operation coordinate C.

[0103] Furthermore, when the identified passing coordinate CSC is within the aerial reception range R, the aerial reception range pass determination unit 215 determines that the pointing input object has passed through the aerial reception range R, in other words, that input has been made into the aerial reception range R.

[0104] FIG. 16 is a flowchart showing the operation of determining whether a pointing input object has passed through one air reception range R in the second embodiment.

[0105] First, the aerial reception range passage judgment unit 215 judges whether the value of the function f at time t-1, that is, f(x(t-1), y(t-1), z(t-1)), is negative, and whether the value of the function f at time t, that is, f(x(t), y(t), z(t)), is positive or 0 (S30).

[0106] This corresponds to the rising edge of f(x(t), y(t), z(t)). This method makes it possible to make the determination because the point (x1, y1, z1) where f(x1, y1, z1)<0 and the point (x2, y2, z2) where f(x2, y2, z2)>0 exist at opposite positions with respect to the surface f(x, y, z)=0. Note that, as in the first embodiment, it is important that the values ​​used for the determination here are those before smoothing, i.e., the three-dimensional operation coordinates C.

[0107] Next, the aerial reception range passage judgment unit 215 judges whether the passage coordinate CSC, which is the smoothed value of the x coordinate, the smoothed value of the y coordinate, and the smoothed value of the z coordinate at time t in the smoothed three-dimensional operation coordinate SC, satisfies the following equations (13) and (14) (S31).

number

number

[0108] If both equations (13) and (14) are satisfied (Yes in S31), the process proceeds to step S32. If at least one of equations (13) and (14) is not satisfied (No in S31), the process proceeds to step S33.

[0109] In step S32, the air reception range passage determination unit 315 determines that the pointing input object has passed through one of the target air reception ranges R. Here, the x and y coordinates of the passage coordinates are given by the following formula (15).

number

[0110] In addition, if multiple aerial reception ranges R are provided, the above flowchart can be performed for all aerial reception ranges R, allowing the aerial reception range pass-through determination unit 315 to identify which aerial reception range R the pointing input object has passed through.

[0111] In the embodiment 2 described above, the aerial reception range R is a part of a sphere, but it can also be implemented in the same way by making the function f(x, y, z) indicating the aerial reception range R have an arbitrary shape with the aerial reception range coordinate system x, y, z as parameters.

[0112] For example, the air reception range R is defined as the center of the air (x c ,y c ,z c ) and the lengths of the x, y, and z axes are x l , y l , z l When forming a rectangular parallelepiped shape such that:

number

[0113] Although not shown in the drawings, when the aerial reception range R is a paraboloid, the function f may be expressed as in the following equation (17).

number

[0114] In this way, f(x, y, z) may be a polynomial such as a hyperboloid, or a shape expressed by a trigonometric polynomial, and since differentiability is not an issue, it may be in a form like a table with parameters of x, y, z or a polynomial combining them, or it may be a combination of two or more planes, a so-called polygon. Note that the aforementioned rectangular parallelepiped is an example of a polygon.

[0115] Also, the first embodiment is a special case included in the second embodiment. That is, the first embodiment is a special case of the second embodiment, in which there are no terms of x and y, such as f(x, y, z) = -z, and the shape is expressed only by the first-order term of z, and g x (x,y,z)=x+C2,g y (x,y,z)=y+C3 (C2 and C3 are constants), so g x is a shape that has no y or z terms and is expressed only by the first order terms of x, g y has no z or x terms, and is expressed only by the first-order term of y.

[0116] Embodiment 3 As shown in FIG. 1, non-contact input device 300 according to embodiment 3 projects an aerial image 102 into the air by a mechanism mounted inside housing 101, similar to non-contact input device 100 according to embodiment 1. Non-contact input device 300 detects operations on buttons 103a, 103b, and 103c included in aerial image 102 using image capture device 104, and notifies through speaker 105 that the operations have been performed.

[0117] As shown in FIG. 2, the non-contact input device 300 in embodiment 3 includes a three-dimensional coordinate acquisition unit 110, a smoothing processing unit 113, an aerial image projection unit 114, an aerial reception range pass-through determination unit 315, a feedback unit 116, and an output unit 117.

[0118] The three-dimensional coordinate acquisition unit 110, smoothing processing unit 113, aerial image projection unit 114, feedback unit 116 and output unit 117 of the non-contact input device 300 in embodiment 3 are similar to the three-dimensional coordinate acquisition unit 110, smoothing processing unit 113, aerial image projection unit 114, feedback unit 116 and output unit 117 of the non-contact input device 100 in embodiment 1.

[0119] In embodiment 3, the aerial reception range pass judgment unit 315 receives three-dimensional operation coordinates C from the three-dimensional operation coordinate detection unit 112, the aerial reception range R from the aerial image projection unit 114, and smoothed three-dimensional operation coordinates SC from the smoothing processing unit 113, provides the pass judgment result P to the feedback unit 116, and identifies the pass coordinates CSC.

[0120] As in the first embodiment, aerial acceptance range pass determination unit 315 determines whether or not a finger, which is an example of an instruction input object, has passed through aerial acceptance range R, based on three-dimensional operation coordinates C and aerial acceptance range R indicated by aerial acceptance range information from aerial image projection unit 114. Then, aerial acceptance range pass determination unit 315 provides a pass determination result P indicating the determination result to feedback unit 116.

[0121] In the third embodiment, aerial acceptance range passage determination unit 315 specifies a passage coordinate CSC indicating which coordinates of aerial acceptance range R a finger, which is an instruction input object, has passed through, based on smoothed three-dimensional operation coordinate SC from smoothing processing unit 113. Here, the passage coordinate CSC specifies, as the passage coordinate CSC at time t, the x-axis value and the y-axis value calculated by the above formula (3) at time t+D, which is a predetermined delay time D from time t at which it is determined that a finger, which is an example of an instruction input object, has passed through aerial acceptance range R, based on three-dimensional operation coordinate C and aerial acceptance range R indicated by aerial acceptance range information from aerial image projection unit 114.

[0122] Furthermore, when multiple aerial acceptance ranges R are set, the aerial acceptance range pass determination unit 315 determines which aerial acceptance range R among the multiple aerial acceptance ranges R that the finger, which is the pointing input object, has passed through, based on the identified pass coordinate CSC and the aerial acceptance range R indicated by the aerial acceptance range information from the aerial image projection unit 114.

[0123] In other words, in the third embodiment, the aerial acceptance range passing determination unit 315 identifies the value of the first axis, which is one of the three axes and extends in a direction intersecting the aerial acceptance range R, from the three-dimensional operation coordinates C, and uses the identified value to identify the time when the pointing input object passed through the aerial acceptance range R. The aerial acceptance range passing determination unit 215 identifies the values ​​of the second axis and the third axis, which are the three axes excluding the first axis, at a delay time that is later than the identified time, as the passing coordinates CSC from the smoothed three-dimensional operation coordinates SC. Then, when the passing coordinates CSC are within the aerial acceptance range R, the aerial acceptance range passing determination unit 215 determines that an input has been made into the aerial acceptance range R.

[0124] Here, the delay time may be any time obtained by adding a time equal to or less than the delay time, which is the time required for the smoothing processor 113 to calculate the smoothed three-dimensional operation coordinates, to the specified time. For example, when the smoothing processing unit 113 performs smoothing using a linear phase filter, the value of the group delay of the linear phase filter may be used as the delay time.

[0125] In the third embodiment, the feedback unit 116 also outputs at the time when the passage determination result P becomes true, similarly to the first embodiment. This allows the user to receive a response to the operation at an early stage.

[0126] Next, the operation of aerial reception range passage determination section 315 in the third embodiment will be described with reference to FIG.

[0127] 18(A) to (C) are graphs for explaining the operation of aerial reception range passage determination section 315 in detail. The graphs in FIGS. 18(A) to (C) are similar to the graphs in FIGS. 8(A) to (C). Here, the delay time D is set to "3", which corresponds to the time for three frames.

[0128] 18(C), at time t=15, z(t), which is the z-axis value in the three-dimensional operation coordinate C, falls, and at this moment, the passing judgment result P becomes true. However, since the delay time D is "3", the aerial reception range passing judgment unit 315 does not identify the passing coordinate CSC at this moment.

[0129] Then, at time t=18, which is obtained by adding a delay time D=3 to time t=15, the aerial reception range pass-through determination unit 315 identifies the x-axis value Vx and y-axis value Vy of the smoothed three-dimensional operation coordinate SC at time t=18 as the pass-through coordinate CSC at time t=15.

[0130] The delay time D is set to "3" in order to cancel the group delay of the 7-tap moving average filter. Therefore, as shown in Figures 18(A) and (C), the value (Vx, Vy) of the passing coordinate CSC at time t = 15 becomes close to the true value (Rx, Ry) at time t = 15.

[0131] Therefore, the third embodiment is effective when it is desired to use a value closer to the true value as the value of the passing coordinate CSC. On the other hand, since the feedback to the user is performed at the moment of reception, i.e., at time t=15, the user feels that the responsiveness is good. However, since the passing coordinate CSC is specified at a time that takes into account the delay time D, if processing using the passing coordinate CSC is required, the actual processing time is not shortened.

[0132] In this case, it is more preferable to set the duration of the feedback to the user to be equal to or longer than the delay time D, since the user is less likely to perceive the feedback as an error. This is because the passing coordinate CSC is identified and substantial processing can be started before the feedback to the user is completed.

[0133] In addition, a moving average filter is used as the smoothing filter here. However, if a smoothing method with nonlinear phase characteristics such as an IIR filter is used, the delay time differs for each frequency component, making it difficult to determine the delay time D.

[0134] For example, FIGS. 19(A) and (B) are schematic diagrams showing the configuration of a two-stage cascade type IIR filter. FIG. 20 is a graph showing the group delay characteristics of the two-stage cascade type IIR filter shown in FIGS. 19(A) and 19(B). Here, FIG. 20 shows a case where the frame rate of the image pickup device 104 is 30 frames per second.

[0135] Thus, the group delay varies depending on the frequency, but is at most 10 samples. In this case, it is appropriate to set the delay time D to a value that is greater than or equal to the minimum value and less than or equal to the maximum value of the group delay characteristic, in this case, greater than or equal to 0 samples and less than or equal to 10 samples. For example, one approach would be to set the delay time D to a time that corresponds to 7 samples, based on the group delay of the DC component. In addition, since this is derived from the sampling rate of the three-dimensional operation coordinate C, it is 30 Hz.

[0136] However, reducing the delay time D also reduces the essential processing time when processing using the passing coordinate CSC is required, so there is also an advantage to setting the delay time D to a value smaller than "7".

[0137] Furthermore, based on this knowledge, even when a filter with linear phase characteristics such as a moving average filter or an FIR filter designed by a window function method is used, it is also advantageous to set the delay time D to a value smaller than the flat group delay. For example, in the case of a 7-tap moving average filter, it is also advantageous to set it to "2" or "1", which is smaller than the group delay of 3 samples. This is effective when it is necessary to balance both the essential processing time and the accuracy of the passing coordinate CSC. When the delay time D is "0", the operation of the third embodiment is the same as that of the first embodiment.

[0138] Embodiment 4 As shown in FIG. 1, non-contact input device 400 according to embodiment 4, like non-contact input device 100 according to embodiment 1, projects an aerial image 102 into the air by a mechanism installed inside housing 101. Non-contact input device 400 detects operations on buttons 103a, 103b, and 103c included in aerial image 102 using image capture device 104, and notifies through speaker 105 that the operations have been performed.

[0139] FIG. 21 is a block diagram showing a schematic configuration of a non-contact input device 400 according to the fourth embodiment. Non-contact input device 400 in embodiment 4 includes a three-dimensional coordinate acquisition unit 110, a smoothing processing unit 113, an aerial image projection unit 114, an aerial reception range pass-through determination unit 415, a feedback unit 116, an output unit 117, and a low-delay smoothing processing unit 418.

[0140] The three-dimensional coordinate acquisition unit 110, smoothing processing unit 113, aerial image projection unit 114, feedback unit 116 and output unit 117 of the non-contact input device 400 in embodiment 4 are similar to the three-dimensional coordinate acquisition unit 110, smoothing processing unit 113, aerial image projection unit 114, feedback unit 116 and output unit 117 of the non-contact input device 100 in embodiment 1.

[0141] The low-latency smoothing processing unit 418 receives three-dimensional operation coordinates C from the three-dimensional operation coordinate detection unit 112, smoothes the three-dimensional operation coordinates C in the time direction using low-latency smoothing processing by the smoothing processing unit 113, and provides the result to the aerial reception range pass-through determination unit 415 as low-latency smoothed three-dimensional operation coordinates LSC. In other words, the low-latency smoothing processing unit 418 calculates a low-latency smoothed three-dimensional operation coordinate LSC by performing smoothing using a second number of three-dimensional operation coordinates, which is a predetermined number equal to or greater than two, included in the multiple three-dimensional operation coordinates C, with a delay time shorter than the time required for the smoothing processing unit 113 to calculate the smoothed three-dimensional operation coordinate SC.

[0142] When the smoothing processing unit 113 performs smoothing using a 7-tap moving average filter as described above, for example, the low-delay smoothing processing unit 418 calculates the low-delay smoothed three-dimensional operation coordinate LSC by performing smoothing using a 3-tap moving average filter as shown in the following equation (18).

number

[0143] The aerial reception range pass judgment unit 415 in embodiment 4 receives low-latency smoothed three-dimensional operation coordinates LSC from the low-latency smoothing processing unit 418, the aerial reception range R from the aerial image projection unit 114, and smoothed three-dimensional operation coordinates SC from the smoothing processing unit 113, provides the pass judgment result P to the feedback unit 116, and identifies the pass coordinates CSC.

[0144] Aerial acceptance range pass determination unit 415 determines whether or not a finger, which is an example of a pointing input object, has passed through aerial acceptance range R, based on low-delay smoothed three-dimensional operation coordinates LSC and aerial acceptance range R indicated by aerial acceptance range information from aerial image projection unit 114. Then, aerial acceptance range pass determination unit 415 provides a pass determination result P indicating the determination result to feedback unit 116.

[0145] For example, similar to the first embodiment, aerial acceptance range passage determination unit 415 identifies a passage coordinate CSC indicating which coordinates in aerial acceptance range R the finger, which is an instruction input object, has passed through, based on smoothed three-dimensional operation coordinate SC from smoothing processing unit 113. Here, the passage coordinate CSC is composed of an x-axis value and a y-axis value calculated by the above formula (3) at time t when it is determined that the finger, which is an example of an instruction input object, has passed through aerial acceptance range R, based on the three-dimensional operation coordinate C and aerial acceptance range R indicated by the aerial acceptance range information from aerial image projection unit 114.

[0146] Furthermore, as in embodiment 1, when multiple aerial acceptance ranges R are set, the aerial acceptance range pass determination unit 415 determines which aerial acceptance range R among the multiple aerial acceptance ranges R that the finger, which is the pointing input object, has passed through, based on the identified pass coordinate CSC and the aerial acceptance range R indicated by the aerial acceptance range information from the aerial image projection unit 114.

[0147] In other words, in the fourth embodiment, the aerial acceptance range passage determination unit 415 specifies the value of the first axis, which is one of the three axes and extends in a direction intersecting the aerial acceptance range R, from the low-delay smoothed three-dimensional operation coordinate LSC, and specifies the time when the pointing input object passed through the aerial acceptance range R using the specified value. The aerial acceptance range passage determination unit 415 specifies the values ​​of the second axis and the third axis, which are the three axes excluding the first axis, at the specified time from the smoothed three-dimensional operation coordinate SC as the passage coordinate CSC. Then, when the passage coordinate CSC is within the aerial acceptance range R, the aerial acceptance range passage determination unit 415 determines that an input has been made into the aerial acceptance range R.

[0148] Here, the low-delay smoothing processor 418 can also perform smoothing using a method other than the 3-tap moving average filter. For example, it is possible to use an FIR filter or an IIR filter using a window function method or the like.

[0149] The above-described three-dimensional operation coordinate detection unit 112, the smoothing processing unit 113, the aerial reception range passage determination unit 415, the feedback unit 116, and the low-delay smoothing processing unit 418 may be partially or entirely configured by a memory 10 and a processor 11 that executes a program stored in the memory 10, as shown in FIG. 6(A). Such a program may be provided through a network, or may be provided by being recorded on a recording medium. That is, such a program may be provided, for example, as a program product.

[0150] In addition, some or all of the three-dimensional operation coordinate detection unit 112, the smoothing processing unit 113, the aerial reception range pass-through determination unit 415, the feedback unit 116 and the low-latency smoothing processing unit 418 can also be configured in the processing circuit 12, for example, as shown in Figure 6 (B). As described above, the three-dimensional operation coordinate detection unit 112, the smoothing processing unit 113, the aerial reception range passage determination unit 415, the feedback unit 116 and the low-delay smoothing processing unit 418 can be realized by a processing circuit network.

[0151] Next, the operation of aerial reception range passage determination section 415 according to the fourth embodiment will be described. FIG. 22 is a flowchart showing the operation of determining whether a pointing input object has passed through one air reception range R in the fourth embodiment. Here, the airborne reception range R is defined as (|x| <x max ,|y| <y max , z=0). Moreover, among the steps in the flowchart shown in FIG. 22, those steps which perform the same processing as the steps in the flowchart shown in FIG. 7 are given the same reference numerals as those in FIG.

[0152] First, the aerial reception range passage determination unit 415 determines whether the value of the z coordinate at time t-1 in the low-latency smoothed three-dimensional operation coordinate LSC is positive or 0, and the value of the z coordinate at time t in the low-latency smoothed three-dimensional operation coordinate LSC is negative (S40). If the condition of step S40 is satisfied (Yes in S40), the process proceeds to step S11, and if the condition of step S40 is not satisfied (No in S40), the process proceeds to step S13. The processing in steps S11 to S13 in FIG. 22 is similar to the processing in steps S11 to S13 in FIG.

[0153] In the first embodiment, the fall is determined using the z-axis value of the unsmoothed three-dimensional operating coordinate C, whereas in the fourth embodiment, the fall is determined using the z-axis value of the low-delay smoothed three-dimensional operating coordinate LSC, which has a relatively short delay time.

[0154] The above operation is shown in detail in FIGS. 23(A) to (C). 23(A) to (C) are graphs for explaining the operation of aerial reception range pass-through determination section 415. In FIG.

[0155] Fig. 23(A) is a graph showing a time series of the x-axis value of the actual finger movement in the air reception range coordinate system, the x-axis value of the three-dimensional operation coordinate C, and the x-axis value of the smoothed three-dimensional operation coordinate SC. The horizontal axis of Fig. 23(A) indicates time, and the vertical axis indicates the x-axis value.

[0156] Fig. 23(B) is a graph showing a time series of the y-axis value of the actual finger movement in the air reception range coordinate system, the y-axis value of the three-dimensional operation coordinate C, and the y-axis value of the smoothed three-dimensional operation coordinate SC. The horizontal axis of Fig. 23(B) indicates time, and the vertical axis indicates the y-axis value.

[0157] Fig. 23(C) is a graph showing a time series of the z-axis value of the actual finger movement in the air reception range coordinate system, the z-axis value of the three-dimensional operation coordinate C, and the z-axis value of the low-latency smoothed three-dimensional operation coordinate LSC. The horizontal axis of Fig. 23(C) indicates time, and the vertical axis indicates the z-axis value.

[0158] In the graphs shown in FIGS. 23(A) to (C), the solid lines indicate true values ​​as actual finger coordinates, and the dashed lines indicate values ​​of the measured three-dimensional operation coordinates C. 23(A) and (B), the double line indicates the value of the smoothed three-dimensional manipulated coordinate SC obtained by smoothing the value of the three-dimensional manipulated coordinate C. On the other hand, in FIG. 23(C), the double line indicates the value of the low-latency smoothed three-dimensional manipulated coordinate LSC obtained by smoothing the value of the three-dimensional manipulated coordinate C with low latency.

[0159] 23(C), the time when the z-axis value in the low-latency smoothed three-dimensional operation coordinate LSC falls from positive to negative is t=16, which is delayed by one frame compared to the z-axis value in the three-dimensional operation coordinate C, but this delay is smaller than the three-frame delay of the z-axis value in the smoothed three-dimensional operation coordinate SC compared to the z-axis value in the three-dimensional operation coordinate C. For this reason, even in the third embodiment, the passage determination result P can be output relatively quickly, and feedback to the user can also be output relatively quickly.

[0160] Furthermore, since the x-axis value and the y-axis value of the smoothed three-dimensional operation coordinate SC can be used as the coordinates passing through the air reception range, values ​​that have been subjected to stronger smoothing can be used. Since the smoothed three-dimensional operation coordinate SC has a relatively large delay, there is a problem that the coordinates of the pointing input object at a past time are output. However, as in the first embodiment, in reality, when the user of the non-contact input device 400 operates the button 103 in the air, there is a tendency to move the finger in a direction perpendicular to the button 103. Therefore, when moving the finger in this direction, the x-axis value and the y-axis value of the smoothed three-dimensional operation coordinate SC are almost constant, so it is considered that the magnitude of this delay is unlikely to be a problem.

[0161] As described above, according to the first to fourth embodiments, smoothed three-dimensional operation coordinates SC are used to detect the coordinates through which the pointing input object passes, so that vibrations due to time-varying noise during detection by the operation input means can be removed.

[0162] However, detection of a pointing input object in the air generally has a low sampling rate, and the delay of the smoothing algorithm is relatively large. For example, if a 3D camera with a frame rate of 30 frames per second is used to detect the position of the pointing input object, and a 7-tap moving average filter is used for smoothing, the delay will be 100 milliseconds. As a result, the system response speed will be slow and the user's operating experience will be poor. Therefore, in embodiments 1 to 4, when determining whether the pointing input object has passed through a plane that includes the range in which operations are accepted, unsmoothed values ​​or low-latency smoothed values ​​are used, thereby making it possible to achieve both system responsiveness and accuracy of the detected position.

[0163] In the above-described first to fourth embodiments, the aerial acceptance range pass judging unit 115 to 415 provides the pass judgment result P to the feedback unit 116 based on the z-axis value of the three-dimensional operation coordinate C or the low-latency smoothed three-dimensional operation coordinate LSC, but the first to fourth embodiments are not limited to such examples. For example, the aerial acceptance range pass judging unit 115 to 415 may provide the pass judgment result P to the feedback unit 116 when the pass coordinate CSC is within the corresponding range. Also, the aerial acceptance range pass judging unit 115 to 415 may provide the pass judgment result P to the feedback unit 116 when the x-axis value and the y-axis value of the three-dimensional operation coordinate C or the low-latency smoothed three-dimensional operation coordinate LSC are within the corresponding range at the time when the z-axis value of the three-dimensional operation coordinate C or the low-latency smoothed three-dimensional operation coordinate LSC falls.

[0164] In the above-described third and fourth embodiments, the aerial acceptance range R is assumed to be a plane for the sake of simplicity of explanation, but as in the second embodiment, the aerial acceptance range R may be part of a shape expressed by a polynomial such as a sphere, a paraboloid, a hyperboloid, or a trigonometric polynomial, and since differentiability is not an issue, it may have a form such as a table with parameters of x, y, z or a polynomial value combining these, or it may be a combination of two or more planes (for example, a rectangular prism or polyhedron), a so-called polygon, as in the second embodiment.

[0165] For example, in the fourth embodiment, the aerial acceptance range passage determination unit 415 determines whether or not the pointing input object has passed through the aerial acceptance range R by using a predetermined function such that the output of the function, which inputs the low-latency smoothed three-dimensional operation coordinate LSC, becomes a predetermined value in the aerial acceptance range R. Here, the aerial acceptance range passage determination unit 415 inputs a plurality of low-latency smoothed three-dimensional operation coordinate LSC to the function in sequence, and determines whether or not the pointing input object has passed through the aerial acceptance range R when the output of the function becomes larger or smaller than that value. [Explanation of symbols]

[0166] 100,200,300,400 Non-contact input device, 101 housing, 102 aerial image, 103 button, 104 image capture device, 105 speaker, 110 three-dimensional coordinate acquisition unit, 111 three-dimensional image capture unit, 112 three-dimensional operation coordinate detection unit, 113 smoothing processing unit, 114 aerial image projection unit, 115,215,315,415 aerial reception range pass determination unit, 116 feedback unit, 117 output unit, 418 low-latency smoothing processing unit.

Claims

1. a three-dimensional operation coordinate detection unit that detects a plurality of three-dimensional operation coordinates by sequentially detecting three-dimensional operation coordinates, which are coordinates of three axes in a predetermined space, of an instruction input object that is an object for inputting an instruction; a smoothing processing unit that calculates smoothed three-dimensional manipulation coordinates by performing smoothing using a first number of three-dimensional manipulation coordinates, the first number being a predetermined number equal to or greater than two, included in the plurality of three-dimensional manipulation coordinates; a low-delay smoothing processing unit that calculates low-delay smoothed three-dimensional manipulation coordinates by performing smoothing using a second number of three-dimensional manipulation coordinates, the second number being a predetermined number of two or more, included in the plurality of three-dimensional manipulation coordinates, with a delay time shorter than a time required for the smoothing processing unit to calculate the smoothed three-dimensional manipulation coordinates; a pass determination unit that uses the low-latency smoothed three-dimensional operation coordinates to determine whether the pointing input object has passed through an air acceptance range that is a predetermined range in the space, and that uses the smoothed three-dimensional operation coordinates to identify pass coordinates that are the coordinates at which the pointing input object has passed through the air acceptance range. A non-contact input device comprising:

2. The passage determination unit determines whether or not the pointing input object has passed through the air acceptance range by using a function that has the low-latency smoothed three-dimensional operation coordinates as an input and that is predetermined so that an output of the function becomes a predetermined value in the air acceptance range.

2. The non-contact input device according to claim 1,

3. The passage determination unit sequentially inputs the plurality of low-latency smoothed three-dimensional operation coordinates to the function, and determines whether or not the pointing input object has passed through the air acceptance range when an output of the function becomes larger or smaller than the value.

3. The non-contact input device according to claim 2, wherein:

4. a three-dimensional operation coordinate detection unit that detects a plurality of three-dimensional operation coordinates by sequentially detecting three-dimensional operation coordinates, which are coordinates of three axes in a predetermined space, of an instruction input object that is an object for inputting an instruction; a smoothing processing unit that calculates smoothed three-dimensional manipulation coordinates by performing smoothing using a first number of three-dimensional manipulation coordinates, the first number being a predetermined number equal to or greater than two, included in the plurality of three-dimensional manipulation coordinates; a pass determination unit that uses the plurality of three-dimensional operation coordinates to determine whether the pointing input object has passed through an air reception range, which is a predetermined range in the space, and that uses the smoothed three-dimensional operation coordinates to identify pass coordinates, which are coordinates at which the pointing input object has passed through the air reception range; When the pointing input object passes through the aerial acceptance range, the passage determination unit specifies a time when the pointing input object passes through the aerial acceptance range, specifies the smoothed three-dimensional operation coordinates at the specified time as the passing coordinates, and determines that an input into the aerial acceptance range has been made when the passing coordinates are within the aerial acceptance range. A non-contact input device comprising:

5. When the pointing input object passes through the aerial acceptance range, the passage determination unit specifies a time when the pointing input object passes through the aerial acceptance range, specifies the smoothed three-dimensional operation coordinates at the specified time as the passing coordinates, and determines that an input into the aerial acceptance range has been made when the passing coordinates are within the aerial acceptance range.

4. The non-contact input device according to claim 1, wherein:

6. a three-dimensional operation coordinate detection unit that detects a plurality of three-dimensional operation coordinates by sequentially detecting three-dimensional operation coordinates, which are coordinates of three axes in a predetermined space, of an instruction input object that is an object for inputting an instruction; a smoothing processing unit that calculates smoothed three-dimensional manipulation coordinates by performing smoothing using a first number of three-dimensional manipulation coordinates, the first number being a predetermined number equal to or greater than two, included in the plurality of three-dimensional manipulation coordinates; a pass determination unit that uses the plurality of three-dimensional operation coordinates to determine whether the pointing input object has passed through an air reception range, which is a predetermined range in the space, and that uses the smoothed three-dimensional operation coordinates to identify pass coordinates, which are coordinates at which the pointing input object has passed through the air reception range; When the pointing input object passes through the air acceptance range, the passage determination unit specifies a time when the pointing input object passes through the air acceptance range, specifies the smoothed three-dimensional operation coordinates at a delay time that is later than the specified time as the passing coordinates, and determines that an input into the air acceptance range has been made when the passing coordinates are within the air acceptance range. A non-contact input device comprising:

7. When the pointing input object passes through the air acceptance range, the passage determination unit specifies a time when the pointing input object passes through the air acceptance range, specifies the smoothed three-dimensional operation coordinates at a delay time that is later than the specified time as the passing coordinates, and determines that an input into the air acceptance range has been made when the passing coordinates are within the air acceptance range.

4. The non-contact input device according to claim 1, wherein:

8. The delay time is a time obtained by adding a time equal to or less than a delay time, which is a time required for the smoothing processing unit to calculate the smoothed three-dimensional operation coordinates, to the specified time.

8. The non-contact input device according to claim 6 or 7,

9. The smoothing processing unit performs the smoothing using a linear phase filter, The delay time is a value of the group delay of the linear phase filter.

9. The non-contact input device according to claim 8,

10. Further comprising an output unit that outputs to the outside of the non-contact input device when the pointing input object passes through the air reception range.

10. The non-contact input device according to claim 1,

11. an output unit that outputs to an outside of the non-contact input device when the pointing input object passes through the air reception range; The output unit continues the output until at least the delay time.

10. The non-contact input device according to claim 6, wherein:

12. The output is sound.

12. The non-contact input device according to claim 10 or 11,

13. The output is ultrasonic waves that give a tactile sensation.

12. The non-contact input device according to claim 10 or 11,

14. a three-dimensional operation coordinate detection unit that detects a plurality of three-dimensional operation coordinates by sequentially detecting three-dimensional operation coordinates, which are coordinates of three axes in a predetermined space, of an instruction input object that is an object for inputting an instruction; a smoothing processing unit that calculates smoothed three-dimensional manipulation coordinates by performing smoothing using a first number of three-dimensional manipulation coordinates, the first number being a predetermined number equal to or greater than two, included in the plurality of three-dimensional manipulation coordinates; a pass determination unit that uses the plurality of three-dimensional operation coordinates to determine whether the pointing input object has passed through an air reception range, which is a predetermined range in the space, and that uses the smoothed three-dimensional operation coordinates to identify pass coordinates, which are coordinates at which the pointing input object has passed through the air reception range; A threshold value for detecting that the pointing input object has passed through the air reception range is different from a threshold value for detecting that the pointing input object has returned to the side before passing through the air reception range after the pointing input object has passed through the air reception range. A non-contact input device comprising:

15. A threshold value for detecting that the pointing input object has passed through the air reception range is different from a threshold value for detecting that the pointing input object has returned to the side before passing through the air reception range after the pointing input object has passed through the air reception range.

14. The non-contact input device according to claim 1,

16. Further comprising an aerial image projection unit that projects an aerial image having a portion corresponding to the aerial reception range into the space.

16. The non-contact input device according to claim 1,

17. A three-dimensional image capturing unit that outputs a three-dimensional image of a predetermined space so that an aerial reception range that is a predetermined range in the predetermined space is included in the space; a three-dimensional operation coordinate detection unit that detects a plurality of three-dimensional operation coordinates by sequentially detecting three-dimensional operation coordinates, which are coordinates of three axes in the space, of an instruction input object that is an object for inputting an instruction from the three-dimensional image; a smoothing processing unit that calculates smoothed three-dimensional manipulation coordinates by performing smoothing using a first number of three-dimensional manipulation coordinates, the first number being a predetermined number equal to or greater than two, included in the plurality of three-dimensional manipulation coordinates; a pass determination unit that determines whether the pointing input object has passed through the air reception range by using the plurality of three-dimensional operation coordinates, and identifies pass coordinates that are the coordinates at which the pointing input object has passed through the air reception range by using the smoothed three-dimensional operation coordinates. A non-contact input device comprising:

18. A three-dimensional image capturing unit outputs a three-dimensional image of the space so that the aerial reception range is included, The three-dimensional operation coordinate detection unit detects the plurality of three-dimensional operation coordinates from the three-dimensional image.

17. The non-contact input device according to claim 1,

19. a three-dimensional operation coordinate detection unit that detects a plurality of three-dimensional operation coordinates by sequentially detecting three-dimensional operation coordinates, which are coordinates of three axes in a predetermined space, of an instruction input object that is an object for inputting an instruction; a smoothing processing unit that calculates smoothed three-dimensional manipulation coordinates by performing smoothing using a first number of three-dimensional manipulation coordinates, the first number being a predetermined number equal to or greater than two, included in the plurality of three-dimensional manipulation coordinates; a pass determination unit that uses the plurality of three-dimensional operation coordinates to determine whether the pointing input object has passed through an air reception range, which is a predetermined range in the space, and that uses the smoothed three-dimensional operation coordinates to identify pass coordinates, which are coordinates at which the pointing input object has passed through the air reception range; The passage determination unit determines whether or not the pointing input object has passed through the air acceptance range by using a function that has the three-dimensional operation coordinates as input and that is predetermined so that an output of the function becomes a predetermined value in the air acceptance range. A non-contact input device comprising:

20. the passage determination unit sequentially inputs the plurality of three-dimensional operation coordinates to the function, and when an output of the function becomes larger or smaller than the value, determines whether or not the pointing input object has passed through the air reception range.

20. The non-contact input device according to claim 19.

21. The pointing input object is a finger.

21. The non-contact input device according to claim 1,

22. The pointing input object is a protrusion.

21. The non-contact input device according to claim 1,

23. Detecting a plurality of three-dimensional operation coordinates by sequentially detecting three-dimensional operation coordinates, which are coordinates of three axes in a predetermined space, of an instruction input object that is an object for inputting an instruction; calculating smoothed three-dimensional manipulation coordinates by performing smoothing using a first number of three-dimensional manipulation coordinates, the first number being a predetermined number equal to or greater than two, which are included in the plurality of three-dimensional manipulation coordinates; Calculating low-delay smoothed three-dimensional manipulation coordinates by performing smoothing using a second number of three-dimensional manipulation coordinates, the second number being a predetermined number equal to or greater than two, included in the plurality of three-dimensional manipulation coordinates, with a delay time shorter than a time required for calculating the smoothed three-dimensional manipulation coordinates; using the low-latency smoothed three-dimensional operation coordinates, determining whether or not the pointing input object has passed through an air reception range, which is a predetermined range in the space; Using the smoothed three-dimensional operation coordinates, a passing coordinate is identified as a coordinate where the pointing input object passes through the air reception range. A non-contact input method comprising:

Images