Information processing device and information processing program

Abstract

To provide an information processing apparatus configured to prevent an instruction from being input to a position different from a position on an operation surface intended by a user when the user performs a non-contact operation on the operation surface by using an operation body, the information processing apparatus being configured so that a position on the operation surface corresponding to a position of the closest portion is defined as an instruction input position by the non-contact operation of the user.SOLUTION: An input determination unit 22 determines, as an instruction input position by a non-contact operation of a user, a position on a display 12 corresponding to a position of the closest portion, which is a portion smallest in distance from the display 12 in a perpendicular direction of the display 12. When a distance between a first position and a second position in a plane direction parallel to the display 12 is equal to or larger than a threshold distance, the first position of the closest portion being detected at a first point of time, and the second position of the closest portion being detected at a second point of time after a minute time has elapsed from the first point of time, the input determination unit 22 determines a determination that the position of the display 12 corresponding to the second position is an instruction input position, to be a false determination.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there have been proposed information processing devices capable of non-contact operation, in which an operation is performed on an operation surface by an operation object such as a finger or a stylus without directly touching the operation surface with the operation object.

[0003] For example, Patent Document 1 discloses an information processing device that changes the display content of a screen as an operation surface by moving a hand that is close to the screen without touching it along the screen, and that determines whether to execute a process to change the display content in accordance with the movement of the hand along the screen based on the tilt of the hand that is close to the screen. Also, Patent Document 2 discloses an information processing device that does not use non-contact operation, but that outputs a warning when a user touches two places on the operation surface of a touch panel at the same time, and suspends processing until the user releases the touch in both places. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-234317 [Patent Document 2] Japanese Patent Application Laid-Open No. 2002-055781 Summary of the Invention [Problem to be solved by the invention]

[0005] In an information processing device having an operation surface, it is conceivable to enable a non-contact operation on the operation surface by detecting a closest part, which is a part of an object in a spatial region facing the operation surface that is the part that is the shortest distance from the operation surface in a direction perpendicular to the operation surface, and determining a position on the operation surface that corresponds to the position of the closest part as an instruction input position by a user's non-contact operation. The spatial region facing the operation surface means a region facing the operation surface in the space near the operation surface, and a region that is the distance from the operation surface in a direction perpendicular to the operation surface within a predetermined distance.

[0006] Consider a case in which a user performs a non-contact operation on an operation surface using an operating object such as a finger or a stylus in an information processing device capable of such contactless operation. Here, if the tip of the operating object is the closest part, the user can input an instruction to the intended position on the operation surface. However, if a part other than the tip of the operating object suddenly becomes the closest part during the contactless operation, the instruction will be input to a position on the operation surface different from the position intended by the user. Examples of parts other than the tip of the operating object include, but are not limited to, a part of the wrist of the user holding the operating object, the user's stomach, or the user's chest.

[0007] The object of the present invention is to prevent instructions from being input at a position on the operation surface other than the position intended by the user when the user is performing a non-contact operation on the operation surface using an operating object in an information processing device in which a position on the operation surface corresponding to the position of the closest part is used as the instruction input position by the user's non-contact operation. [Means for solving the problem]

[0008] The invention of claim 1 is an information processing device comprising a processor, which detects a closest part, which is a part of an object in a spatial region opposite an operation surface that is the part that is the shortest distance from the operation surface in a direction perpendicular to the operation surface, determines a position on the operation surface corresponding to the position of the closest part as an instruction input position by a user's non-contact operation, and when a distance in a plane parallel to the operation surface between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after a small amount of time has elapsed from the first time point is equal to or greater than a threshold distance, invalidates an instruction input to a position on the operation surface that corresponds to the second position. The invention of claim 2 is an information processing device comprising a processor, wherein the processor detects a closest part, which is a part of an object in a spatial region opposite an operation surface that is the part that is the shortest distance from the operation surface in a direction perpendicular to the operation surface, determines a position on the operation surface that corresponds to the position of the closest part as an instruction input position by a user's non-contact operation, and outputs a warning to the user when a distance in a plane parallel to the operation surface between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after a short time has elapsed from the first time point is equal to or greater than a threshold distance. The invention of claim 3 is the information processing device described in claim 1 or 2, characterized in that when the distance in a plane parallel to the operation surface between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after a small amount of time has elapsed from the first time point is equal to or greater than a threshold distance, the processor determines that a position on the operation surface corresponding to the second position is the instruction input position as an erroneous determination. The invention of claim 4 is the information processing device described in claim 3, characterized in that when the second position is located on the opposite side of the first position from the direction in which the user is estimated to be located, the processor does not consider it an erroneous judgment that the position on the operation surface corresponding to the second position is the instruction input position. The invention of claim 5 is the information processing device described in claim 4, characterized in that when the operation surface is upright or when the operation surface is tilted so as to face vertically upward and horizontally, the downward direction along a plane direction parallel to the operation surface relative to the first position is the direction in which the user is estimated to be located. The invention of claim 6 is the information processing device described in claim 3, characterized in that when instructions can be input to the operation surface by gestures and the second position is located in a direction defined by the gesture relative to the first position, it is not an erroneous determination that the position on the operation surface corresponding to the second position is the instruction input position. The invention according to claim 7 is the information processing device according to claim 1 or 2, characterized in that the threshold distance can be set for each user. The invention of claim 8 is the information processing device described in claim 7, characterized in that the processor determines the threshold distance for each user depending on the movement speed of the operating object for each user's past non-contact operation. The invention of claim 9 is the information processing device according to claim 3, wherein the processor determines that a position on the operation surface corresponding to a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after a small amount of time has elapsed since the first time point is equal to or greater than the threshold distance, and determines that a position on the operation surface corresponding to the second position is the instruction input position as an erroneous determination, and determines that a position on the operation surface corresponding to the position of the closest part is the instruction input position as an erroneous determination, and determines that a position on the operation surface corresponding to the position of the closest part is the instruction input position as an erroneous determination, and determines that a position on the operation surface corresponding to the position of the closest part is the instruction input position as an erroneous determination, and The invention of claim 10 is an information processing program that causes a computer to detect a closest part, which is a part of an object in a spatial region facing an operation surface that is the part that is closest to the operation surface in a direction perpendicular to the operation surface, determines a position on the operation surface corresponding to the position of the closest part as an instruction input position by a user's non-contact operation, and invalidates an instruction input to a position on the operation surface that corresponds to the second position when a distance in a plane parallel to the operation surface between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after a short time has elapsed from the first time point is equal to or greater than a threshold distance. The invention of claim 11 is an information processing program that causes a computer to detect a closest part, which is a part of an object in a spatial region opposite an operation surface that is the part that is the shortest distance from the operation surface in a direction perpendicular to the operation surface, determine a position on the operation surface that corresponds to the position of the closest part as an instruction input position by a user's non-contact operation, and output a warning to the user when the distance in a plane parallel to the operation surface between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after a short time has elapsed from the first time point is equal to or greater than a threshold distance. [Effects of the Invention]

[0009] According to the inventions of claims 1 to 3 or 10 to 11, in an information processing device in which a position on the operation surface corresponding to the position of the closest part is set as the instruction input position by the user's non-contact operation, when the user is performing a non-contact operation on the operation surface using an operating object, it is possible to prevent instructions from being input at a position on the operation surface other than the position intended by the user. According to the invention of claim 4 or 5, even when the second position is located on the opposite side of the direction in which the user is estimated to be located from the first position, the accuracy of the erroneous determination can be improved compared to when the position on the operation surface corresponding to the second position is determined to be the instruction input position. According to the invention of claim 6, it is possible to prevent a gesture from being invalidated or a warning from being output as a result of making a gesture. According to the invention of claim 7, it is possible to invalidate an instruction input to a position on the operation surface corresponding to the second position, or to output a warning to the user, based on a threshold distance corresponding to each user. According to the invention of claim 8, it is possible to set a threshold value according to the operation speed of each user's non-contact operation. According to the invention of claim 9, the determination of the instruction input position can be made erroneous by different methods depending on the distance between the first position and the second position in a plane direction parallel to the operation surface. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic diagram illustrating the configuration of an information processing device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a diagram showing a spatial region facing a display. [Figure 3] FIG. 10 is a diagram showing the distance in the XpYp plane between a first position and a second position. [Figure 4] FIG. 1 is a first diagram showing a non-detection region. [Figure 5] FIG. 2 is a second diagram showing the non-detection region. [Figure 6] FIG. 3 is a third diagram showing a non-detection region. [Figure 7] FIG. 4 is a fourth diagram showing a non-detection region. [Figure 8] FIG. 5 is a fifth diagram showing a non-detection region. [Figure 9] FIG. 10 is a diagram showing an example of a state in which an object other than the operating object is the closest part. [Figure 10] FIG. 10 is a diagram showing a threshold distance and a second threshold distance. [Figure 11] 10 is a flowchart showing a flow of processing performed by the information processing device according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] FIG. 1 is a schematic diagram of the configuration of an information processing device 10 according to this embodiment. As will be described in detail later, the information processing device 10 is a device capable of non-contact operation. Non-contact operation refers to performing an operation on an operation surface (display 12 in this embodiment) using an operation object without directly touching the operation surface with the operation object. The operation object is used to perform non-contact operation, and is, for example, a user's finger or a stylus. Note that non-contact operation is also called hover operation.

[0012] In this embodiment, the information processing device 10 is a multifunction device that has a print function, a copy function, a scan function, etc. and performs a print process, a copy process, a scan process, etc. in response to a processing command (job) from a user, but the information processing device 10 is not limited to this and may be any device as long as it allows contactless operation.

[0013] 1, the information processing device 10 includes a display 12, an object sensor 14, a memory 16, and a processor 18. Although not shown in Fig. 1, the information processing device 10 may also include a communication interface (e.g., a network interface card (NIC)) for communicating with other devices via a communication line such as a local area network (LAN) or a wide area network (WAN), a touch operation interface (e.g., a button or a touch panel) for receiving instructions from a user through a touch operation, and a processing device (e.g., a printer or a scanner) for executing a printing process or a scanning process.

[0014] The display 12 includes, for example, a liquid crystal panel or an organic EL (Electro Luminescence) panel. Various screens are displayed on the display 12 by the processor 18. For example, the display 12 displays a contactless operation screen including operation icons that are the target of contactless operation. In this embodiment, the display surface of the display 12 corresponds to the operation surface that is the target of contactless operation.

[0015] The object sensor 14 is a sensor that detects an object that is not in contact with the display surface (i.e., the operation surface, hereinafter simply referred to as the display 12) of the display 12 but is approaching the display 12. More specifically, the object sensor 14 detects the presence or absence of an object approaching the display 12 and the position of the object approaching the display 12. The position of the object includes the position in a plane parallel to the display 12 and the position in a direction perpendicular to the display 12. The object sensor 14 detects not only an operating object for performing a non-contact operation on the display 12, but also any object approaching the display 12.

[0016] Various known methods can be used to detect an object. For example, object sensor 14 may be a capacitance sensor that detects a change in capacitance between display 12 and an object. In this case, processor 18 can detect the presence and position of an object based on the change in capacitance between display 12 and the object detected by object sensor 14. Alternatively, object sensor 14 may be an optical sensor that detects light. In this case, a light source (not shown) emits infrared or laser light toward the display surface of display 12, and object sensor 14 detects the reflected light, particularly the light reflected from the object. Processor 18 can detect the presence and position of an object based on the reflected light detected by object sensor 14.

[0017] A detection signal indicating the detection of an object and the location of the detected object is transmitted from the object sensor 14 to a processor 18 .

[0018] The memory 16 includes a hard disk drive (HDD), a solid state drive (SSD), an embedded multi-media card (eMMC), a read-only memory (ROM), a random access memory (RAM), or the like. The memory 16 stores an information processing program for operating each unit of the information processing device 10. The information processing program can also be stored in a computer-readable non-transitory storage medium such as a universal serial bus (USB) memory or a CD-ROM. The information processing device 10 can read and execute the information processing program from such a storage medium.

[0019] The processor 18 refers to a processor in a broad sense and includes at least one of a general-purpose processor (e.g., a central processing unit (CPU)) and a dedicated processing device (e.g., a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a programmable logic device). The processor 18 may not be a single processing device, but may be configured by the cooperation of multiple processing devices located in physically separate locations. As shown in FIG. 1, the processor 18 performs the functions of an object detection unit 20, an input determination unit 22, a threshold distance setting unit 24, and a notification processing unit 26 in accordance with an information processing program stored in the memory 16.

[0020] Based on the detection signal from the object sensor 14, the object detection unit 20 detects the closest part of the object in the spatial region opposite the display 12, which is the part that is the shortest distance from the display 12 in the direction perpendicular to the display 12.

[0021] The processing of the object detection unit 20 will be described in detail with reference to Fig. 2. Fig. 2 is a diagram showing a spatial region 30 facing the display 12 and an operating object 32 at least a part of which is located in the spatial region 30. In Fig. 2 (as in Figs. 3 to 10), the direction parallel to the display 12 (the horizontal direction of the display 12) is the Xp axis, the direction parallel to the display 12 and perpendicular to the Xp axis (the vertical direction of the display 12) is the Yp axis, and the direction perpendicular to the display 12 is the Zp axis.

[0022] The spatial region 30 facing the display 12 is a region through which the display 12 passes when the display 12 is translated in the positive direction of the Zp axis, and is a region whose distance from the display 12 in the Zp axis direction is within a predetermined distance. Note that the predetermined distance, i.e., the length of the spatial region 30 in the Zp axis direction, may be determined according to the detectable range of the object sensor 14. In other words, the predetermined distance may be the distance at which the object sensor 14 can detect an object.

[0023] As shown in FIG. 2, when an object (operating object 32 in the example of FIG. 2) approaches display 12, object sensor 14 detects the object. Specifically, object sensor 14 detects each part of the object (for example, the tip of the index finger or the tip of the thumb of operating object 32) and transmits a detection signal indicating the position of each part of the object to processor 18. Based on the detection signal from object sensor 14, object detection unit 20 compares the distance Lv from display 12 in the Zp axis direction of each part in space region 30. Then, of the parts in space region 30, the part with the smallest distance Lv is detected as the closest part.

[0024] 2, part 32a, which is the tip of the index finger, and part 32b, which is the tip of the thumb, of the operating body 32 are shown as representative parts of an object in the spatial region 30 detected by the object sensor 14. The object detection unit 20 compares the distance Lva in the Zp axis direction between part 32a and the display 12, the distance Lvb in the Zp axis direction between part 32b and the display 12, and the distance Lv in the Zp axis direction between other parts of the object in the spatial region 30 and the display 12, and because distance Lva is the smallest, detects part 32a corresponding to distance Lva as the closest part.

[0025] Since an object in the spatial region 30 may move, the object detection unit 20 detects the nearest part at unit time intervals based on the detection signal from the object sensor 14. Here, the unit time interval is, for example, an interval of several milliseconds or less.

[0026] The input determination unit 22 determines position A on the display 12 corresponding to the position of the closest part (part 32a in the example of FIG. 2) detected by the object detection unit 20 as the pointing input position by the user's non-touch operation. Position A on the display 12 corresponding to the position of the closest part is a point on the display 12 having the same Xp coordinates and Yp coordinates as the closest part. In other words, if the position of the closest part is expressed by coordinates (Xp, Yp, Zp)=(x, y, z) in the XpYpZp space, the pointing input position will be coordinates (Xp, Yp)=(x, y) on the display 12.

[0027] For example, if an operation icon associated with some processing is displayed at the instruction input position, input determination unit 22 can determine that a non-touch operation has been performed on the operation icon by determining the instruction input position. In this case, from the viewpoint of preventing erroneous operations on the operation icon, input determination unit 22 may determine that a non-touch operation has been performed on the operation icon when it determines that the instruction input position has been within the display area of ​​the operation icon for a predetermined operation confirmation time. Processor 18 executes processing associated with the operation icon on which the non-touch operation has been performed.

[0028] Alternatively, the input determination unit 22 may be capable of determining an operation by a gesture. In other words, the user may be able to input instructions by a gesture on the display 12. A gesture refers to the movement of the operating body 32, or in other words, the movement (movement pattern) of the instruction input position. Of course, in this embodiment, the user moves the operating body 32 in the spatial region 30 without touching the display 12 with the operating body 32, thereby moving the instruction input position and inputting an instruction by a gesture. The movement pattern of the instruction input position and the processing content are associated in advance, and the input determination unit 22 can detect the movement pattern of the instruction input position (in other words, the movement pattern of the closest part detected by the object detection unit 20) and determine the instruction content according to the detected movement pattern. The processor 18 executes the processing associated with the identified gesture.

[0029] In addition to the above, the processor 18 can execute various processes based on the pointing input position determined by the input determination unit 22.

[0030] While a user is performing a non-contact operation on the display 12 using the operating object 32, the object detection unit 20 may detect a part other than the tip of the operating object 32 as the closest part, contrary to the user's intention. For example, as shown in FIG. 2, when the user is performing a non-contact operation using a finger as the operating object 32, the object detection unit 20 detects part 32a, the tip of the index finger, as the closest part as intended by the user at the start of the non-contact operation. However, if the user changes the angle of their arm, for example, and the position of the operating object 32 becomes as shown in FIG. 3, the distance Lvc in the Zp axis direction between part 32c, which is the wrist, and the display 12 becomes smaller than the distance Lva in the Zp axis direction between part 32a and the display 12, and part 32c may be detected as the closest part. In this case, position C corresponding to part 32c, the closest part, becomes the pointing input position, rather than position A intended by the user.

[0031] As described above, when a part other than the tip of the operating object 32 is detected as the closest part against the user's intention while the user is performing a non-contact operation on the display 12 using the operating object 32, the position of the closest part changes significantly within a very short time. Here, a very short time means a very short time, for example, of a few tens of milliseconds or less. For example, in the example of FIG. 3, when the user is performing a non-contact operation with the index finger as the operating object 32, the tip of the index finger 32a may move slightly due to the user's operation, but it is unlikely that the position of the part 32a will move to the position of the wrist 32c within a very short time.

[0032] Therefore, when the distance in a plane direction parallel to display 12 between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after an infinitesimal time has elapsed since the first time point is equal to or greater than the threshold distance, input determination unit 22 determines that the determination that the position on display 12 corresponding to the second position is the pointing input position is an erroneous determination. In other words, input determination unit 22 does not regard the position on display 12 corresponding to the second position as the pointing input position.

[0033] A specific description will be given with reference to Fig. 3. As a premise, the input determination unit 22 detects the nearest part based on the detection signal from the object sensor 14 at a predetermined movement determination time interval. As described above, the object detection unit 20 detects the nearest part at unit time intervals, and the movement determination time may be a unit time, but does not necessarily have to be a unit time. For example, the movement determination time may be a time equivalent to a plurality of unit times. However, the movement determination time is a very short time (preferably tens of milliseconds or less).

[0034] Assume that at a first time point, the object detection unit 20 determines that the part 32a, which is the tip of the index finger, is the closest part. At this time, the input determination unit 22 stores information indicating the position of the part 32a, in particular the Xp coordinate and Yp coordinate of the part 32a. Thereafter, assume that the user changes the posture of the operating object 32 as shown in FIG. 3, and at a second time point after a movement determination time has elapsed from the first time point, the object detection unit 20 determines that the part 32c, which is the wrist, is the closest part. At this time, the input determination unit 22 acquires the Xp coordinate and Yp coordinate of the part 32c, and calculates the distance in a plane direction parallel to the display 12 (i.e., the XpYp plane direction) between the position of the part 32a, which is the closest part at the first time point, and the part 32c, which is the closest part at the second time point.

[0035] In this embodiment, the input determination unit 22 calculates a distance Lh between a position A on the display 12 corresponding to the position of the portion 32a, which is the closest portion at a first time point, and a position C on the display 12 corresponding to the position of the portion 32c, which is the closest portion at a second time point. In other words, the input determination unit 22 calculates the distance between the pointing input position at the first time point and the pointing input position at the second time point. Note that the distance between the portion 32a and the portion 32c in the plane direction parallel to the display 12 does not necessarily have to be calculated using the distance between the position A and the position C on the display 12, and the distance in the XpYp plane direction between the portion 32a and the portion 32c in the spatial domain 30 may be directly calculated.

[0036] Then, the input determination unit 22 compares the calculated distance Lh with a predetermined threshold distance, and if the distance Lh is equal to or greater than the threshold distance, determines that the determination that the position C is the pointing input position is an erroneous determination. Note that if the distance Lh is less than the threshold distance, the input determination unit 22 determines the position C as the pointing input position. Here, the threshold distance may be set in advance by an administrator of the information processing device 10 or the like, or may be set by a threshold distance setting unit 24 (described in detail later).

[0037] If the input determination unit 22 determines that the position on the display 12 corresponding to the second position is the instruction input position as an erroneous determination, it can invalidate the instruction input for that instruction input position. Alternatively, if the input determination unit 22 determines that the position on the display 12 corresponding to the second position is the instruction input position as an erroneous determination, it can output a warning to the user via the notification processing unit 26 (described in detail later).

[0038] If, while a user is performing a non-contact operation on the display 12 using the operating body 32, a part other than the tip of the operating body 32 is detected as the closest part, contrary to the user's intention, the second position (part 32c in the example of Figure 3) is often located in the direction of the user rather than the first position (part 32a in the example of Figure 3).

[0039] Therefore, the input determination unit 22 may determine, as a non-detection area, an area of ​​the spatial area 30 facing the display 12 on the opposite side of the direction in which the user is estimated to be located with respect to the first position, and when the second position is located in the non-detection area, may not erroneously determine that the position on the display 12 corresponding to the second position is the pointing input position. In other words, only when the second position is located in the direction in which the user is estimated to be located with respect to the first position (an area other than the non-detection area in the spatial area 30), may the input determination unit 22 determine, as an erroneous determination, that the position on the display 12 corresponding to the second position is the pointing input position.

[0040] The input determination unit 22 may determine the relationship between the first position and the second position based on the positions on the display 12 corresponding to the first position and the second position. That is, the input determination unit 22 may determine, as a non-detection surface area, a surface area on the display 12 opposite to the direction in which the user is estimated to be located, with respect to the position on the display 12 corresponding to the first position, so that when the position on the display 12 corresponding to the second position is in the non-detection surface area, the determination that the position on the front operation surface corresponding to the second position is the instruction input position is not erroneously determined.

[0041] The non-detection surface area may be, for example, a surface area where an operator such as an operation icon is displayed. That is, the non-detection area may be, for example, an area in space facing an area where an operator such as an operation icon is displayed.

[0042] The direction in which the user is estimated to be present is, for example, the direction in which the user's torso is estimated to be present. Alternatively, the direction in which the user is estimated to be present is the direction in which the user's arm is estimated to extend from the tip of the operating body 32 toward the user's torso. The direction in which the user is estimated to be present can be set in advance.

[0043] For example, consider a case where the display 12 is attached to the front side of the information processing device 10, as shown in Fig. 4. In Fig. 4 (and similarly in Figs. 5, 7, and 8), the horizontal direction in real space is the Xs axis, the horizontal direction perpendicular to the Xs axis is the Ys axis, and the vertical direction is the Zs axis. In Fig. 4, the Xs axis is the lateral direction of the information processing device 10, and the Ys axis is the front-to-rear direction of the information processing device 10. Also, in Fig. 4, the display 12 is installed on a horizontal plane, and the Xp axis and the Xs axis, the Yp axis and the Ys axis, and the Zp axis and the Zs axis are aligned, respectively.

[0044] When the display 12 is attached to the front side of the information processing device 10, the user usually operates the information processing device 10 from the front of the information processing device 10, and therefore, it can be said that the user is often in front of the information processing device 10. Therefore, in this case, the front of the display 12 (the front of the information processing device 10) can be set in advance as the direction in which the user is estimated to be present. Therefore, when the position of the part 32a shown in FIG. 4 is detected as the first position, the area of ​​the spatial region 30 behind the position of the part 32a becomes the non-detection region 30a (shaded portion).

[0045] FIG. 5 is a perspective view of the display 12. As shown in FIG. 5, consider a case where the display 12 is installed upright or a case where the display 12 is tilted so as to face vertically upward (the positive direction of the Zs axis) and horizontally (the positive direction of the Ys axis in FIG. 5). In this case, the extension direction of the user's arm from the tip of the operating body 32 (the user's fingertip or the tip of a stylus held by the user) toward the user's torso is often downward (the positive direction of the Yp axis) along a plane parallel to the display 12 (the direction of the XpYp plane). Therefore, in this case, the downward direction (the positive direction of the Yp axis) along a plane parallel to the display 12 can be set in advance as the direction in which the user is estimated to be present. Therefore, when the position of the portion 32a shown in FIG. 5 is detected as the first position, the region of the spatial region 30 on the negative side of the Yp axis (upper side along the Yp axis) of the position of the portion 32a becomes the non-detection region 30a.

[0046] FIG. 6 is a plan view of the display 12. Consider the case where characters are displayed on the display 12 as shown in FIG. 6. In this case, the direction in which the user's arm extends from the tip of the operating body 32 toward the user's torso is often opposite to the upright direction of the characters (here, the direction is assumed to be from the bottom to the top of the upright characters). Therefore, in this case, the direction opposite to the upright direction of the characters can be set in advance as the direction in which the user is estimated to be present. Therefore, when the position of the portion 32a shown in FIG. 6 is detected as the first position, the region of the spatial region 30 that is closer to the upright direction of the characters than the position of the portion 32a becomes the non-detection region 30a.

[0047] The information processing device 10 may be a table-shaped device, in which a display 12 is provided on the table surface and non-contact operation of the display 12 is possible from multiple directions. FIGS. 7 and 8 are plan views of the display 12 of such an information processing device 10. In such a display 12, the input determination unit 22 may determine the direction in which the user is estimated to be present based on the detected position of the first position. When the direction in which the user is estimated to be present changes, the non-detection area may be dynamically changed.

[0048] For example, assume that the display 12 is intended to be viewed from two directions, the positive Yp-axis direction (lower side of FIG. 7 ) and the negative Yp-axis direction (upper side of FIG. 7 ), as shown in FIG. 7 . In this case, for example, when the input determination unit 22 detects a first position on the positive Yp-axis side of the center of the spatial region 30 in the Yp-axis direction, as indicated by the dashed-dotted line in FIG. 7 , the positive Yp-axis direction is determined to be the direction in which the user is presumed to be present. Therefore, when the position of the part 32a shown in FIG. 7 is detected as the first position, the region of the spatial region 30 on the negative Yp-axis side of the position of the part 32a becomes the non-detection region 30a. On the other hand, when the position of the part 32a shown in FIG. 8 is detected as the first position, the direction in which the user is presumed to be present is the negative Yp-axis direction (upper side of FIG. 8 ), and therefore the region of the spatial region 30 on the positive Yp-axis side of the position of the part 32a becomes the non-detection region 30a.

[0049] In this way, by dividing the spatial region 30 into a plurality of subspace regions and associating each subspace region with a direction in which the user is estimated to be located when the first position is detected within that subspace region in advance, the input determination unit 22 can suitably determine the direction in which the user is estimated to be located based on the first position, even in cases where the display 12 of the table-shaped information processing device 10 is expected to be viewed from three or more directions.

[0050] In addition, if the information processing device 10 has a sensor that detects a user around the information processing device 10 (for example, a camera that takes pictures of the area around the information processing device 10), the input determination unit 22 may obtain the direction in which the user is located based on the detection signal of the sensor.

[0051] As described above, when the input determination unit 22 is capable of determining an operation by a gesture, there may be a case where the user moves the operating body 32 at high speed in the spatial region 30 to perform a gesture operation. In this case, even if a part other than the tip of the operating body 32 is not detected as the closest part, the distance in a plane direction parallel to the display 12 between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point may be equal to or greater than the threshold distance. In this case, the determination that the second position corresponds to the instruction input position may be erroneously determined, which may result in a problem that the gesture operation is not properly recognized or a warning is erroneously output.

[0052] Therefore, when the second position is located in a direction defined by the gesture relative to the first position, the input determination unit 22 may prevent an erroneous determination that a position on the display 12 corresponding to the second position is the pointing input position. For example, suppose a gesture is defined in the spatial domain 30 to quickly move the operating object 32 in the Xp direction. In this case, when the second position is located in the Xp axis direction of the first position, the input determination unit 22 prevents an erroneous determination that a position on the display 12 corresponding to the second position is the pointing input position, even if the distance between the first position and the second position in a plane parallel to the display 12 is equal to or greater than a threshold distance. Note that the case where the second position is located in the Xp axis direction of the first position includes not only a case where the Yp coordinate of the first position and the Yp coordinate of the second position are identical, but also a case where the difference between the Yp coordinate of the first position and the Yp coordinate of the second position is within a predetermined range. The direction defined by the gesture may be set separately for each screen displayed on the display 12.

[0053] However, before the user performs a non-contact operation using the operating object 32, a part of the user (such as the abdomen, chest, or clothing) or a part of another object may accidentally enter and remain in the spatial region 30 without the user's intention. For example, as shown in FIG. 9 , when the user attempts to perform a non-contact operation on the display 12 with a finger as the operating object 32, a part 32d, which is the user's abdomen, may enter and remain in the spatial region 30. In this case, the part 32d may be detected as the closest part, and a position D on the display 12 corresponding to the position of the part 32d, which is not intended by the user, becomes the pointing input position. Furthermore, if the distance Lvd of the part 32d from the display 12 in the Zp axis direction is smaller than the distance Lva of the part 32a, the user will not be able to set the position A corresponding to the position of the part 32a as the pointing input position.

[0054] When a part of an object not intended by the user (part 32d in the example of FIG. 9) remains in the spatial region 30, the position of the part 32d often does not move much. Therefore, when the input determination unit 22 continues to detect the closest part within a local region within the spatial region 30 for a threshold time, it is preferable that the input determination unit 22 determines that the position on the display 12 corresponding to the position of the closest part is the pointing input position as an erroneous determination. The local region here does not mean an area at a specific position within the spatial region 30, but means an area having a predetermined size. The size of the local region may be predetermined by an administrator of the information processing device 10, etc. The threshold time may also be predetermined by an administrator of the information processing device 10, etc.

[0055] Specifically, input determination unit 22 detects the nearest part at a predetermined movement determination time interval and stores the position of the nearest part at each time point. If it is determined that the positions of multiple nearest parts detected within a threshold time are all within a certain range of area (i.e., within a local area), it determines that the position on display 12 corresponding to the position of the nearest part is an erroneous determination as the pointing input position.

[0056] When the input determination unit 22 continues to detect, for the threshold time, that the distance in a plane direction parallel to the display 12 between the position of the closest part at the previous time point and the position of the closest part at the next time point after the movement determination time from the previous time point is less than the second threshold distance, the input determination unit 22 determines that the closest part has been detected in the local region within the spatial region 30 for the threshold time. In other words, when the input determination unit 22 continues to detect, for the threshold time, that the distance between the instruction input position at the previous time point and the instruction input position at the next time point is less than the second threshold distance, the input determination unit 22 determines that the closest part has been detected in the local region within the spatial region 30 for the threshold time.

[0057] In this embodiment, the input determination unit 22 determines that it has continued to detect the nearest part within a local region within the spatial region 30 for the threshold time if it continues to detect, for the threshold time, that the distance between the position on the display 12 corresponding to the position of the nearest part at the previous time point and the position on the display 12 corresponding to the position of the nearest part at the next time point after the movement determination time from the previous time point is less than a second threshold distance.

[0058] Specifically, the input determination unit 22 increments a counter each time the distance between the instruction input position at the previous time point and the instruction input position at the next time point becomes less than the second threshold distance, and when the counter reaches a predetermined value (because the nearest part is detected at the movement determination time interval, the fact that the counter has reached the predetermined value means that the distance between the instruction input position at the previous time point and the instruction input position at the next time point has remained less than the second threshold distance for the threshold time), the input determination unit 22 determines that the nearest part has continued to be detected in the local region within the spatial region 30 for the threshold time. If the distance between the instruction input position at the previous time point and the instruction input position at the next time point becomes equal to or greater than the second threshold distance during the threshold time, the input determination unit 22 clears the counter (returns it to 0). The second threshold distance here may be set in advance by an administrator of the information processing device 10, for example.

[0059] In addition, the input determination unit 22 may calculate the distance between the first detected instruction input position within the threshold time and each of the multiple instruction input positions detected during the threshold time, and if all of the calculated multiple distances are less than a second threshold distance, determine that the closest part has continued to be detected within a local region within the spatial region 30 for the threshold time.

[0060] In addition, the distance in a plane parallel to the display 12 between the closest point at the previous time point and the closest point at the next time point does not necessarily have to be calculated based on the distance between the pointing input position at the previous time point and the pointing input position at the next time point, but may be calculated directly as the distance in the XpYp plane direction between the closest point at the previous time point and the closest point at the next time point in the spatial domain 30.

[0061] If the input determination unit 22 determines that the position on the display 12 corresponding to the position of the closest part is the instruction input position as an erroneous determination, it can invalidate the instruction input for that instruction input position. Alternatively, if the input determination unit 22 determines that the position on the display 12 corresponding to the position of the closest part is the instruction input position as an erroneous determination, it can output a warning to the user by the notification processing unit 26 (described in detail later).

[0062] As described above, the input judgment unit 22 can perform a process (referred to as a first erroneous judgment process) in which, when the distance in a plane parallel to the display 12 between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after a small amount of time has elapsed since the first time point is a threshold distance or longer, it is determined that a position on the display 12 corresponding to the second position is an erroneous judgment as a pointing input position; and a process (referred to as a second erroneous judgment process) in which, when the closest part continues to be detected within a local region within the spatial region 30 for a threshold time, it is determined that a position on the display 12 corresponding to the position of the closest part is an erroneous judgment as a pointing input position.

[0063] Here, the input determination unit 22 may be capable of selecting and executing an appropriate process from a first erroneous determination process and a second erroneous determination process depending on the distance in a plane parallel to the display 12 between a first position, which is the position of the closest part at a first time point, and a second position, which is the position of the closest part at a second time point. Specifically, when the distance in a plane parallel to the display 12 between the first position of the closest part detected at the first time point and the second position of the closest part detected at a second time point after an infinitesimal time has elapsed from the first time point, is equal to or greater than a threshold distance, the input determination unit 22 may determine that a position on the display 12 corresponding to the second position is the pointing input position as an erroneous determination; and when the distance in the plane parallel to the display 12 between the first position and the second position is less than the threshold distance and the closest part has been detected continuously in a local region within the spatial region 30 for the threshold time, the input determination unit 22 may determine that a position on the display 12 corresponding to the position of the closest part is the pointing input position as an erroneous determination.

[0064] A specific description will be given with reference to Figure 10. Figure 10 is a diagram showing the threshold distance and the second threshold distance as viewed from the Zp axis direction. Point F represents the position of the closest part at a first time point, i.e., the first position. D1 is the threshold distance referenced in the first error determination process, and the position that is the threshold distance D1 away from the first position F is represented by the outer circle. D2 is the second threshold distance referenced in the second error determination process, and the position that is the second threshold distance D2 away from the first position F is represented by the inner circle.

[0065] Assume that the position of the closest part at the second time point, that is, the second position, is the position indicated by reference symbol S1 shown in Fig. 10. In this case, the distance between the first position F and the second position S1 in a plane parallel to the display 12 (the XpYp plane) is equal to or greater than the threshold distance D1, and therefore the input determination unit 22 determines, through the first error determination process, that the determination that the position on the display 12 corresponding to the second position S1 is the pointing input position is an error.

[0066] Assume that the second position is the position indicated by reference symbol S2 in FIG. 10 . In this case, the distance between the first position F and the second position S2 in a plane parallel to the display 12 is less than the threshold distance D1 and less than the second threshold distance D2. Therefore, the input determination unit 22 starts the second erroneous determination process. That is, when the input determination unit 22 determines that the closest part has been continuously detected in a local region within the spatial region 30 for the threshold time, the input determination unit 22 determines that the position on the display 12 corresponding to the closest part is the pointing input position as an erroneous determination. As described above, in this embodiment, when the distance between the first position F and the second position S2 in a plane parallel to the display 12 is less than the second threshold distance D2, the input determination unit 22 then determines whether the distance between the second position S2 and the position of the closest part at a third time point after the movement determination time from the second time point is less than the second threshold distance D2. This process is repeated for each movement determination time, and if it is determined that the closest part has been detected continuously within a local area within the spatial area 30 for a threshold time, the determination that the position on the display 12 corresponding to the position of the closest part is the instruction input position is determined to be an erroneous determination.

[0067] If the second position is the position indicated by symbol S3 in Figure 10, that is, if the distance in a plane parallel to the display 12 between the first position F and the second position S3 is less than the threshold distance D1 and greater than or equal to the second threshold distance D2, the input judgment unit 22 does not consider the judgment that the position on the display 12 corresponding to the second position S3 is the instruction input position to be an erroneous judgment, but rather considers the instruction input position to be valid.

[0068] Returning to FIG. 1 , the threshold distance setting unit 24 sets the above-mentioned threshold distance. The threshold distance may be set as a parameter common to all users, or the threshold distance setting unit 24 may set the threshold distance for each user. For example, the threshold distance setting unit 24 sets the threshold distance for each user in response to an instruction from each user, and stores in the memory 16 a user ID that identifies the user and the threshold distance of the user in association with each other. The user ID can be obtained by the processor 18 authenticating the user prior to a non-contact operation on the display 12.

[0069] Furthermore, the threshold distance setting unit 24 may determine the threshold distance for each user according to the moving speed of the operating object 32 for non-contact operation of each user in the past. Specifically, the threshold distance setting unit 24 calculates the moving speed of the operating object 32 based on the position of the closest part of the operating object 32 detected by the object detection unit 20 at unit time intervals. Then, the threshold distance setting unit 24 sets a longer threshold distance for a user with a faster moving speed and a shorter threshold distance for a user with a slower moving speed. This makes it possible to improve the accuracy of determining that a position on the display 12 corresponding to the second position is an instruction input position, based on the moving speed of the operating object 32 for non-contact operation of each user.

[0070] The notification processing unit 26 performs processing to notify the user of the determination result of the input determination unit 22. For example, the notification processing unit 26 causes the display 12 to display a pointer indicating the instruction input position determined by the input determination unit 22.

[0071] Furthermore, when the input determination unit 22 determines that the position on the display 12 corresponding to the second position is an instruction input position, the notification processing unit 26 outputs a warning to the user. When the notification processing unit 26 outputs a warning to the user, the input determination unit 22 may or may not invalidate the instruction input to the position on the display 12 corresponding to the second position.

[0072] The warning may be a warning that notifies the user that there has been an erroneous determination of the instruction input position. For example, the notification processing unit 26 displays a message on the display 12 to the effect that there has been an erroneous determination of the instruction input position. The warning may also be a warning that notifies the user of the position on the display 12 that corresponds to the second position. For example, the notification processing unit 26 displays a pointer in a different display mode from normal (for example, a different color or flashing) at the position on the display 12 that corresponds to the second position. This allows the user to grasp the second position.

[0073] Furthermore, instead of or in addition to displaying the warning on the display 12, the notification processing unit 26 may output the warning by a method such as sound or vibration, or by a combination of at least two of display, sound, and vibration.

[0074] The configuration of the information processing device 10 according to this embodiment is as described above. The flow of processing by the information processing device 10 according to this embodiment will be described below with reference to the flowchart shown in FIG.

[0075] In step S10, the object detection unit 20 determines whether or not an object has been detected in the spatial region 30 facing the display 12, which is the operation surface that is the target of the non-contact operation, based on the detection signal from the object sensor 14. Here, it is assumed that an object has been detected in the spatial region 30, and the process proceeds to step S12.

[0076] In step S12, the object detection unit 20 detects the closest part of the object in the spatial region 30, which is the part that is the shortest distance from the display 12 in the direction perpendicular to the display 12 (Zp axis direction), and obtains the current position, which is that position.

[0077] In step S14, the input determination unit 22 determines whether or not the previous position, which is the position of the closest part detected at the previous point in time, is held. Here, it is assumed that the previous position is not held, and the process proceeds to step S16.

[0078] In step S16, the input determination unit 22 stores the current position of the closest part acquired in step S12 as the previous position in the memory 16. Here, the previous position becomes the first position. In addition, the notification processing unit 26 displays a pointer at a position on the display 12 corresponding to the previous position.

[0079] In step S18, the input determination unit 22 waits for a movement determination time, which is a very short time.

[0080] In step S10 again, the object detection unit 20 determines whether or not an object has been detected in the spatial region 30. If no object is detected, that is, if the object including the closest portion detected in the previous step S12 has moved outside the spatial region 30, the process proceeds to step S20. In step S20, the input determination unit 22 erases the previous position stored in the previous step S16 from the memory 16. In addition, the notification processing unit 26 hides the pointer that was displayed in the previous step S16. Here, it is determined that an object has been detected in the spatial region 30 again in step S10, and the process proceeds to step S12.

[0081] In step S12 again, the object detection unit 20 detects the closest part again and acquires its current position, which becomes the second position.

[0082] In step S14 again, the previous position is held, so the process proceeds to step S21.

[0083] In step S21, the input determination unit 22 determines whether the current position is within the non-detection area 30a. If the current position is within the non-detection area 30a, the process proceeds to step S16, and if the current position is not within the non-detection area 30a, the process proceeds to step S22.

[0084] In step S22, the input determination unit 22 determines whether the distance between the previous position (here, the first position) and the current position (here, the second position) in a plane direction parallel to the display 12 is equal to or greater than a threshold distance D1. If the distance is equal to or greater than the threshold distance D1, the process proceeds to step S24.

[0085] In step S24, the input determination unit 22 determines that the determination that the position on the display 12 corresponding to the current position is the instruction input position is an erroneous determination. Then, the input determination unit 22 invalidates the instruction input to the position on the display 12 corresponding to the current position, or the notification processing unit 26 outputs a warning to the user.

[0086] In step S22, if the distance between the previous position and the current position in the plane direction parallel to the display 12 is less than the threshold distance D1, the process proceeds to step S26.

[0087] In step S26, the input determination unit 22 determines whether the distance between the previous position and the current position in a plane direction parallel to the display 12 is less than the second threshold distance D2. If the distance is not less than the second threshold distance D2, the process proceeds to step S28. In step S28, the input determination unit 22 clears the counter (a counter for determining that the closest part has been detected continuously within the local region in the spatial region 30 for the threshold time), and returns to step S16.

[0088] In step S26, if the distance between the previous position and the current position in a plane parallel to the display 12 is less than the second threshold distance D2, the input determination unit 22 proceeds to step S30. In step S30, the input determination unit 22 increments the counter by 1. Note that the counter is cleared at the start of this flowchart.

[0089] In step S32, the input determination unit 22 determines whether the counter has reached a predetermined value N. If the counter has reached the predetermined value N, the input determination unit 22 determines that the closest part has been detected within the local region within the spatial region 30 for the threshold time, and proceeds to step S24. If the counter has not reached the predetermined value N, the process returns to step S16, and the processes of steps S16, S18, S10, S12, S14, S22, S26, and S30 are repeated.

[0090] Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications are possible without departing from the spirit of the present invention. [Explanation of symbols]

[0091] 10 information processing device, 12 display, 14 object sensor, 16 memory, 18 processor, 20 object detection unit, 22 input determination unit, 24 threshold distance setting unit, 26 notification processing unit, 30 spatial region, 30a non-detection region, 32 operating body, 32a, 32b, 32c, 32d parts.

Claims

1. a processor; The processor: detecting a closest portion, which is a portion of an object that is in a spatial region facing the operation surface and that is the portion that is the shortest distance from the operation surface in a direction perpendicular to the operation surface; determining a position on the operation surface corresponding to the position of the closest part as an instruction input position by a non-touch operation by the user; when a distance in a plane direction parallel to the operation surface between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after an elapse of a small time period from the first time point is equal to or greater than a threshold distance, invalidating an instruction input to a position on the operation surface corresponding to the second position; 1. An information processing device comprising:

2. a processor; The processor: detecting a closest portion, which is a portion of an object that is in a spatial region facing the operation surface and that is the portion that is the shortest distance from the operation surface in a direction perpendicular to the operation surface; determining a position on the operation surface corresponding to the position of the closest part as an instruction input position by a non-touch operation by the user; a warning is output to the user when a distance in a plane parallel to the operation surface between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after an elapse of a small time period from the first time point is equal to or greater than a threshold distance; 1. An information processing device comprising:

3. The processor: When a distance in a plane direction parallel to the operation surface between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after an elapse of a small time period from the first time point is equal to or greater than a threshold distance, a determination that a position on the operation surface corresponding to the second position is the instruction input position is determined to be an erroneous determination.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

4. The processor: When the second position is located on the opposite side of the first position from the direction in which the user is estimated to be present, a determination that the position on the operation surface corresponding to the second position is the instruction input position is not an erroneous determination.

4. The information processing apparatus according to claim 3,

5. When the operation surface is erected or when the operation surface is tilted so as to face vertically upward and horizontally, a downward direction along a plane direction parallel to the operation surface with respect to the first position is a direction in which the user is estimated to be present.

5. The information processing apparatus according to claim 4,

6. An instruction can be input by gesture on the operation surface, When the second position is located in a direction defined by the gesture with respect to the first position, a determination that a position on the operation surface corresponding to the second position is the instruction input position is not an erroneous determination.

4. The information processing apparatus according to claim 3,

7. The threshold distance can be set for each user.

3. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

8. The processor: determining the threshold distance for each user according to a moving speed of an operating object for a non-contact operation by each user in the past; 8. The information processing apparatus according to claim 7,

9. The processor: when a distance in a plane direction parallel to the operation surface between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after an elapse of an infinitesimal time from the first time point is equal to or greater than the threshold distance, determining that a position on the operation surface corresponding to the second position is the instruction input position as an erroneous determination, when a distance between the first position and the second position in a plane direction parallel to the operation surface is less than the threshold distance and the closest part has been continuously detected within a local region within the spatial region for a threshold time, a determination that a position on the operation surface corresponding to the position of the closest part is the instruction input position is an erroneous determination.

4. The information processing apparatus according to claim 3,

10. On the computer, detecting a closest portion of an object that is in a spatial region facing the operation surface, the closest portion being a portion that is the smallest distance from the operation surface in a direction perpendicular to the operation surface; determining a position on the operation surface corresponding to the position of the closest part as an instruction input position by a non-touch operation by the user; when a distance in a plane direction parallel to the operation surface between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after an elapse of a small time period from the first time point is equal to or greater than a threshold distance, invalidating an instruction input to a position on the operation surface corresponding to the second position; An information processing program characterized by:

11. On the computer, detecting a closest portion of an object that is in a spatial region facing the operation surface, the closest portion being a portion that is the smallest distance from the operation surface in a direction perpendicular to the operation surface; determining a position on the operation surface corresponding to the position of the closest part as an instruction input position by a non-touch operation by the user; outputting a warning to the user when a distance in a plane parallel to the operation surface between a first position of the closest part detected at a first time point and a second position of the closest part detected at a second time point after an elapse of a short time from the first time point is equal to or greater than a threshold distance; An information processing program characterized by:

Images