Information processing device and information processing program

Abstract

To provide an information processing apparatus in which a position on an operation surface corresponding to a position of the closest portion is defined as an instruction input position by a non-contact operation of a user, configured to prevent an instruction not intended by the user from being input.SOLUTION: An input determination unit 22 determines, as an instruction input position for a non-contact operation of a user, a position on a display 12 corresponding to a position of the closest portion, which is smallest in distance from the display 12 in a perpendicular direction of the display 12, of portions of an object located in a spatial region 30 that faces the display 12. When continuously detecting the closest portion for a threshold time within a local region in the spatial region 30, the input determination unit 22 determines that the position on the display 12 corresponding to the position of the closest portion is the instruction input position, to be a false determination.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an information processing program.

Background Art

[0002] Conventionally, there has been proposed an information processing apparatus capable of performing a non-contact operation in which an operation on an operation surface is performed by an operation body without directly touching the operation surface with the operation body such as a finger or a stylus.

[0003] For example, Patent Document 1 discloses an information processing apparatus that changes the display content of a screen by moving a hand that is brought close to the screen along the screen without contacting the screen as an operation surface, and determines whether to execute a process of changing the display content according to the movement of the hand along the screen based on the inclination of the hand brought close to the screen. Further, Patent Document 2 discloses an information processing apparatus that, although not by non-contact operation, outputs a warning when a user touches two locations on an operation surface at the same time on a touch panel and suspends the process until the user releases the touches at the two locations.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] Incidentally, in an information processing device having an operating surface, it is conceivable to enable non-contact operation of the operating surface by detecting the nearest point of an object in the spatial region opposite the operating surface, which is the point with the smallest distance from the operating surface in the direction perpendicular to the operating surface, and determining the position on the operating surface corresponding to the position of the nearest point as the user's non-contact instruction input position. The spatial region opposite the operating surface refers to the region in the vicinity of the operating surface that is opposite to the operating surface, and the region where the distance from the operating surface in the direction perpendicular to the operating surface is within a predetermined distance.

[0006] In such contactless information processing devices, it is conceivable that a part of the user's body, or a part of another object, might accidentally enter the spatial area opposite the operating surface and remain within that area. In such a case, an unintended instruction could be input to the information processing device.

[0007] The objective of the present invention is to suppress the input of unintended instructions by the user in an information processing device in which a position on the operating surface corresponding to the position of the nearest contact point is used as an instruction input position by the user through non-contact operation. [Means for solving the problem]

[0008] The invention according to claim 1 is an information processing device comprising a processor, the processor detecting the nearest point of contact which is the part of an object in a spatial region opposite to the operating surface that is the part that is the closest to the operating surface in the direction perpendicular to the operating surface, determining the position on the operating surface corresponding to the position of the nearest point of contact as the user's non-contact input position, and invalidating the input to the position on the operating surface corresponding to the position of the nearest point of contact if the nearest point of contact is continuously detected for a threshold time within a local region in the spatial region. The invention according to claim 2 is an information processing device comprising a processor, the processor detecting the nearest point of contact which is the part of an object in a spatial region opposite to the operating surface that is the part that is the closest to the operating surface in the direction perpendicular to the operating surface, determining the position on the operating surface corresponding to the position of the nearest point of contact as the user's non-contact input position, and outputting a warning to the user if the nearest point of contact is continuously detected for a threshold time within a local region in the spatial region. The invention according to claim 3 is an information processing device according to claim 1 or 2, characterized in that when the processor continues to detect the nearest neighbor for a threshold time within a local region in the spatial region, it incorrectly determines that the position on the operating surface corresponding to the position of the nearest neighbor is the instruction input position. The invention according to claim 4 is an information processing device according to claim 3, characterized in that if the processor continues to detect the nearest neighbor for a threshold time within the local region which is part of the spatial region, the determination that the position on the operating surface corresponding to the position of the nearest neighbor is the instruction input position is not considered an incorrect determination. The invention according to claim 5 is an information processing device according to claim 4, characterized in that the non-detection area is an area located on the opposite side from the direction in which the user is presumed to be located. The invention according to claim 6 is an information processing device according to claim 4, characterized in that, when the operating surface is erected, or when the operating surface is inclined to face vertically upward and horizontally, the downward direction along the planar direction parallel to the operating surface is the direction in which the user is presumed to be located. The invention according to claim 7 is an information processing device according to claim 3, characterized in that the processor stores the misjudgment position, which is the position of the nearest neighbor when the misjudgment occurred, in memory, sets the threshold time for each of a plurality of subspace regions within the spatial region such that the threshold time becomes shorter the more times the misjudgment position is detected within the subspace region, and determines that the position on the operating surface corresponding to the position of the nearest neighbor is the instruction input position if the nearest neighbor is continuously detected within the local region within the subspace region for the threshold time set for the subspace region, and determines that this position is the instruction input position. The invention according to claim 8 is an information processing apparatus according to claim 7, characterized in that the processor stores the misjudgment locations in memory for each user and sets the threshold time for each of the plurality of subspace regions for each user. The invention according to claim 9 is that the processor is This is the position of the nearest neighbor detected at the first time point. First position and , the position of the nearest neighbor detected at the second time point, a small time interval after the first time point. Distance in the plane parallel to the operating surface between the second position and the second position is the threshold If the distance is less than the specified value and the nearest neighbor is continuously detected within the local area for a threshold time, the determination that the position on the operating surface corresponding to the position of the nearest neighbor is the instruction input position will be considered an incorrect determination. The aforementioned First position and The aforementioned The distance between the second position and the operating surface in a planar direction parallel to the operating surface is The aforementioned The information processing device according to claim 3, characterized in that if the distance is greater than or equal to a threshold distance, the determination that the position on the operating surface corresponding to the second position is the instruction input position is incorrect. The invention according to claim 10 is an information processing program characterized by causing a computer to detect the nearest point, which is the part of an object in a spatial region opposite to the operating surface that is the part that is the closest to the operating surface in the direction perpendicular to the operating surface; causing the computer to determine the position on the operating surface corresponding to the position of the nearest point as the user's non-contact input position; and if the computer continues to detect the nearest point within a local region in the spatial region for a threshold time, it invalidates the input of an instruction for the position on the operating surface corresponding to the position of the nearest point. The invention according to claim 11 is an information processing program characterized by causing a computer to detect the nearest point of an object in a spatial region opposite to the operating surface, which is the point that is the point that is the closest to the operating surface in the direction perpendicular to the operating surface; causing the computer to determine the position on the operating surface corresponding to the position of the nearest point as the user's non-contact input position; and outputting a warning to the user if the nearest point continues to be detected in a local region within the spatial region for a threshold time. [Effects of the Invention]

[0009] According to the invention of claims 1 to 3 or 10 to 11, in an information processing device in which a position on the operating surface corresponding to the position of the nearest contact point is used as a non-contact input position for user operation, it is possible to suppress the input of instructions that the user did not intend. According to the invention of claim 4, even when the nearest neighbor is continuously detected for a threshold time within the local area located within the non-detection area, the accuracy of the misjudgment can be improved compared to the case where the determination that the position on the operating surface corresponding to the position of the nearest neighbor is the instruction input position is misjudged. According to the invention of claim 5 or 6, if the nearest neighbor is continuously detected for a threshold time in a non-detection area located on the opposite side of the direction in which the user is presumed to be located, the position on the operating surface corresponding to the position of the nearest neighbor can be set as the instruction input position. According to the invention of claim 7, different threshold times can be set for each subspace region. According to the invention of claim 8, different threshold times can be set for each subspace region for each user. According to the invention of claim 9, the determination of the instruction input position can be misjudged by different methods depending on the distance between the first position and the second position in a planar direction parallel to the operating surface. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram of the configuration of the information processing device according to this embodiment. [Figure 2] It is a diagram showing a space area facing the display. [Figure 3] It is a diagram showing an example of a state where an object other than the operating body is the closest part. [Figure 4] It is the first diagram showing the non-detection area. [Figure 5] It is the second diagram showing the non-detection area. [Figure 6] It is the third diagram showing the non-detection area. [Figure 7] It is the fourth diagram showing the non-detection area. [Figure 8] It is the fifth diagram showing the non-detection area. [Figure 9] It is a diagram showing the distance in the XpYp plane between the first position and the second position. [Figure 10] It is a diagram showing the threshold distance and the second threshold distance. [Figure 11] It is a diagram showing a partial space area. [Figure 12] It is a flowchart showing the processing flow of the information processing apparatus according to the present embodiment.

Embodiments for Carrying Out the Invention

[0011] FIG. 1 is a schematic configuration diagram of an information processing apparatus 10 according to the present embodiment. As will be described in detail later, the information processing apparatus 10 is a device capable of non-contact operation. The non-contact operation is to perform an operation on the operation surface by the operation body without directly touching the operation body on the operation surface (in this embodiment, the display 12). The operation body is for performing a non-contact operation, and for example, it is a user's finger or a stylus. Note that the non-contact operation is also called a hover operation.

[0012] In the present embodiment, the information processing apparatus 10 is a multifunction device having a print function, a copy function, a scan function, etc., and executing print processing, copy processing, or scan processing, etc. according to a processing instruction (job) from a user. However, the information processing apparatus 10 is not limited to this, and any device may be used as long as non-contact operation is possible.

[0013] As shown in Figure 1, the information processing device 10 comprises a display 12, an object sensor 14, a memory 16, and a processor 18. Although not shown in Figure 1, the information processing device 10 may also include a communication interface (e.g., a NIC (Network Interface Card)) for communicating with other devices via communication lines such as a LAN (Local Area Network) or WAN (Wide Area Network), a contact operation interface (e.g., buttons or a touch panel) for receiving instructions from the user through contact operation, and a processing device (e.g., a printer or scanner) for performing print and scan operations.

[0014] The display 12 is composed of, for example, a liquid crystal panel or an organic EL (Electro-Luminescence) panel. Various screens are displayed on the display 12 by the processor 18. For example, the display 12 displays a contactless operation screen that includes operation icons that are the target of contactless operation. In this embodiment, the display surface of the display 12 corresponds to the operation surface that is the target of contactless operation.

[0015] The object sensor 14 is a sensor that detects objects approaching the display 12, even though they are not in contact with the display surface (i.e., the operating surface, hereafter simply referred to as the display 12). More specifically, the object sensor 14 detects the presence or absence of an object approaching the display 12, and the position of the object approaching the display 12. The position of the object includes its position in a plane parallel to the display 12 and its position perpendicular to the display 12. The object sensor 14 detects not only operating objects used for non-contact operation of the display 12, but also any object that approaches the display 12.

[0016] Various known methods can be used to detect objects. For example, the object sensor 14 may be a capacitive sensor that detects changes in capacitance between the display 12 and the object. In this case, the processor 18 can detect the presence and position of the object in accordance with the changes in capacitance between the display 12 and the object detected by the object sensor 14. Alternatively, the object sensor 14 may be an optical sensor that detects light. In this case, infrared light or laser light is emitted from a light source (not shown) in the direction of the display surface of the display 12, and the object sensor 14 detects the reflected light, especially the reflected light from the object. The processor 18 can detect the presence and position of the object based on the reflected light detected by the object sensor 14.

[0017] A detection signal indicating that an object has been detected and the location of the detected object is transmitted from the object sensor 14 to the processor 18.

[0018] Memory 16 is composed of components such as an HDD (Hard Disk Drive), SSD (Solid State Drive), eMMC (embedded Multi Media Card), ROM (Read Only Memory), or RAM (Random Access Memory). Information processing programs for operating each part of the information processing device 10 are stored in memory 16. These information processing programs can also be stored on a computer-readable non-temporary storage medium such as a USB (Universal Serial Bus) memory or CD-ROM. The information processing device 10 can read and execute these information processing programs from such storage media.

[0019] The processor 18 refers to a processor in a broad sense and consists of at least one of the following: a general-purpose processor (e.g., a CPU (Central Processing Unit)) and a dedicated processing unit (e.g., a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a programmable logic device). The processor 18 may not consist of a single processing unit, but rather of multiple processing units located in physically separate locations working together. As shown in Figure 1, the processor 18 performs its functions as an object detection unit 20, an input determination unit 22, a threshold time setting unit 24, and a notification processing unit 26 according to the information processing program stored in the memory 16.

[0020] The object detection unit 20 detects the nearest part of an object in the spatial region facing the display 12, based on the detection signal from the object sensor 14, which is the part that is closest to the display 12 in the direction perpendicular to the display 12.

[0021] The processing of the object detection unit 20 will be explained in detail with reference to Figure 2. Figure 2 shows a spatial region 30 facing the display 12, and an operating body 32 in which at least a part is located within the spatial region 30. In Figure 2 (and similarly in Figures 3 to 11), the direction parallel to the display 12 (the horizontal direction of the display 12) is defined as the Xp axis, the direction parallel to the display 12 and perpendicular to the Xp axis (the vertical direction of the display 12) is defined as the Yp axis, and the direction perpendicular to the display 12 is defined as the Zp axis.

[0022] The spatial region 30 opposite the display 12 is the region that the display 12 passes through when the display 12 is translated in the positive direction of the Zp axis, and is a region within a predetermined distance from the display 12 in the Zp axis direction. The predetermined distance, that is, the length of the spatial region 30 in the Zp axis direction, may be determined according to the detectable range of the object sensor 14. In other words, the predetermined distance may be the detectable distance of an object by the object sensor 14.

[0023] As shown in Figure 2, when an object (operator 32 in the example of Figure 2) approaches the display 12, the object sensor 14 detects the object. Specifically, the object sensor 14 detects each part of the object (for example, the tip of the index finger or the tip of the thumb of the operator 32) and transmits a detection signal indicating the position of each part of the object to the processor 18. Based on the detection signals from the object sensor 14, the object detection unit 20 compares the distance Lv from the display 12 in the Zp axis direction for each part in the spatial region 30. Then, it detects the part with the smallest distance Lv among the parts in the spatial region 30 as the nearest point.

[0024] In the example shown in Figure 2, the parts of the object in the spatial region 30 detected by the object sensor 14 are typically shown as the tip of the index finger 32a and the tip of the thumb 32b of the operating body 32. The object detection unit 20 compares the distance Lva in the Zp axis direction between part 32a and the display 12, the distance Lvb in the Zp axis direction between part 32b and the display 12, and the distance Lv in the Zp axis direction between other parts of the object in the spatial region 30 and the display 12. Since distance Lva is the smallest, part 32a corresponding to distance Lva is detected as the nearest part.

[0025] Since objects in the spatial domain 30 can move, the object detection unit 20 detects the nearest point based on the detection signal from the object sensor 14 at unit time intervals. Here, the unit time interval is, for example, a few milliseconds or less.

[0026] The input determination unit 22 determines that the position A on the display 12 corresponding to the position of the nearest object detected by the object detection unit 20 (object 32a in the example of Figure 2) is the instruction input position by the user's non-contact operation. Position A on the display 12 corresponding to the position of the nearest object is a point on the display 12 with the same Xp and Yp coordinates as the nearest object. That is, if the position of the nearest object is represented by the coordinates (Xp,Yp,Zp)=(x,y,z) in the XpYpZp space, then the instruction input position will be the coordinates (Xp,Yp)=(x,y) on the display 12.

[0027] For example, if an operation icon associated with some process is displayed at the instruction input position, the input determination unit 22 can determine that a non-contact operation has been performed on that operation icon by determining the instruction input position. In this case, from the viewpoint of suppressing erroneous operations on the operation icon, the input determination unit 22 may determine that a non-contact operation has been performed on the operation icon if it determines that the instruction input position remains within the display area of ​​the operation icon for a predetermined operation confirmation time. The processor 18 executes the process associated with the operation icon on which the non-contact operation was performed.

[0028] Alternatively, the input determination unit 22 may be capable of determining operation by gesture. In other words, the user may be able to input instructions by gesture to the display 12. A gesture means the movement of the operating body 32, or in other words, the movement (operation pattern) of the instruction input position. Of course, in this embodiment, the user moves the operating body 32 in the spatial area 30 without touching the display 12, thereby moving the instruction input position and inputting instructions by gesture. The operation pattern of the instruction input position and the processing content are associated in advance, and the input determination unit 22 can detect the operation pattern of the instruction input position (in other words, the operation pattern of the nearest object detected by the object detection unit 20) and determine the instruction content according to the detected operation pattern. The processor 18 executes the processing associated with the identified gesture.

[0029] In addition to the above, the processor 18 can perform various processes based on the instruction input position determined by the input determination unit 22.

[0030] Incidentally, before a user performs a non-contact operation using the operating body 32, there are cases where a part of the user (such as the abdomen, chest, or clothing) or another part of an object unintentionally enters and remains in the spatial region 30. For example, as shown in Figure 3, when a user attempts to perform a non-contact operation on the display 12 with their finger as the operating body 32, a part of the user's abdomen, 32c, may enter the spatial region 30 and remain there. In this case, part 32c may be detected as the nearest point of contact, and the position C on the display 12 corresponding to the position of part 32c, which the user did not intend, becomes the instruction input position. Furthermore, if the distance Lvc of part 32c from the display 12 in the Zp axis direction is smaller than the distance Lva of part 32a, the user will be unable to set the position A corresponding to the position of part 32a as the instruction input position.

[0031] If a part of an object unintended by the user (part 32c in the example of Figure 3) remains in the spatial region 30, the position of part 32c often does not move much. Therefore, if the input determination unit 22 continues to detect the nearest part within a local region in the spatial region 30 for a threshold time, it incorrectly determines that the position on the display 12 corresponding to the position of the nearest part is the instruction input position. Here, the local region refers not to a region at a specific location within the spatial region 30, but to a region with a predetermined size. The size of the local region may be predetermined by the administrator of the information processing device 10 or the like. The threshold time may also be predetermined by the administrator of the information processing device 10 or the like. Furthermore, the threshold time may be set by the threshold time setting unit 24 (details described later).

[0032] As a prerequisite, the input determination unit 22 detects the nearest point based on the detection signal from the object sensor 14 at predetermined movement determination time intervals. As described above, the object detection unit 20 detects the nearest point at unit time intervals, and the movement determination time may be a unit time, but it does not necessarily have to be a unit time. For example, the movement determination time may be a time equivalent to multiple unit times.

[0033] The input determination unit 22 detects the nearest point at predetermined movement determination time intervals and stores the position of the nearest point at each time point. If it is determined that the positions of multiple nearest points detected within a threshold time are all within a certain range (i.e., within a local area), the determination that the position on the display 12 corresponding to the position of the nearest point should be the instruction input position is incorrect.

[0034] The input determination unit 22 determines that the nearest point has been continuously detected within the local area of ​​the spatial area 30 for a threshold time if it continuously detects for a threshold time that the distance in the planar direction parallel to the display 12 between the position of the nearest point at the previous time and the position of the nearest point at the next time after the movement determination time from the previous time is less than the threshold distance. In other words, the input determination unit 22 determines that the nearest point has been continuously detected within the local area of ​​the spatial area 30 for a threshold time if it continuously detects for a threshold time that the distance between the instruction input position at the previous time and the instruction input position at the next time is less than the threshold distance.

[0035] In this embodiment, the input determination unit 22 determines that the nearest neighbor has been continuously detected within the local area of ​​the spatial area 30 for a threshold time if it continues to detect for a threshold time that the distance between the position on the display 12 corresponding to the position of the nearest neighbor at the previous time and the position on the display 12 corresponding to the position of the nearest neighbor at the next time after the movement determination time from the previous time is less than the threshold distance.

[0036] Specifically, the input determination unit 22 increments a counter each time the distance between the instruction input position at the previous time and the instruction input position at the next time falls below the threshold distance. When the counter reaches a predetermined value (since the nearest point is detected at the movement determination time interval, the counter reaching a predetermined value means that the distance between the instruction input position at the previous time and the instruction input position at the next time remained below the threshold distance for the threshold time), the unit determines that the nearest point has been continuously detected within the local area of ​​the spatial area 30 for the threshold time. If the distance between the instruction input position at the previous time and the instruction input position at the next time becomes greater than or equal to the threshold distance during the threshold time, the input determination unit 22 clears the counter (resets it to 0). The threshold distance here may be set in advance by the administrator of the information processing device 10 or the like.

[0037] Furthermore, the input determination unit 22 may calculate the distance between the first instruction input position detected within the threshold time and each of the multiple instruction input positions detected during the threshold time, and if all of the calculated distances are less than the threshold distance, it may determine that the nearest neighbor has been continuously detected within the local region of the spatial region 30 for the threshold time.

[0038] Furthermore, the distance in the plane parallel to the display 12 between the nearest point at the previous time and the nearest point at the next time does not necessarily have to be calculated using the distance between the instruction input position at the previous time and the instruction input position at the next time. The distance in the XpYp plane direction between the nearest point at the previous time and the nearest point at the next time in the spatial region 30 may be calculated directly.

[0039] If the input determination unit 22 incorrectly determines that the position on the display 12 corresponding to the location of the nearest point is the instruction input position, it can invalidate the instruction input for that instruction input position. Alternatively, if the input determination unit 22 incorrectly determines that the position on the display 12 corresponding to the location of the nearest point is the instruction input position, the notification processing unit 26 (details described later) can output a warning to the user.

[0040] Prior to a user performing a non-contact operation on the display 12 using the operating body 32, any object that happens to remain in the spatial area 30 is often located in the direction of the user. Furthermore, as in this embodiment, when the operating surface for non-contact operation is the display 12, if operation icons or messages are displayed on the display 12, the user may focus on these, and the operating body 32 may remain in the area of ​​the spatial area 30 opposite to the operation icon or message.

[0041] In light of this, the input determination unit 22 should define a non-detection area in a part of the spatial area 30, and if the nearest neighbor is continuously detected for a threshold time within a local area within the non-detection area, it should not incorrectly determine that the position on the display 12 corresponding to the position of the nearest neighbor is the instruction input position. In other words, it should only incorrectly determine that the position on the display 12 corresponding to the position of the nearest neighbor is the instruction input position if the nearest neighbor is continuously detected for a threshold time within a local area of ​​the spatial area 30 other than the non-detection area.

[0042] The input determination unit 22 can designate the area of ​​the spatial region 30 that is located on the opposite side from the direction in which the user is presumed to be located as a non-detection area.

[0043] The direction in which the user is presumed to be located is, for example, the direction in which the user's torso is presumed to be located. Alternatively, the direction in which the user is presumed to be located is the direction in which the user's arm extends from the tip of the operating body 32 toward the user's torso.

[0044] For example, consider the case where the display 12 is mounted on the front side of the information processing device 10, as shown in Figure 4. In Figure 4 (and similarly in Figures 5, 7, 8, and 11), the horizontal direction in real space is defined as the Xs axis, the horizontal direction perpendicular to the Xs axis is defined as the Ys axis, and the vertical direction is defined as the Zs axis. In Figure 4, the Xs axis is the lateral direction of the information processing device 10, and the Ys axis is the front-to-back direction of the information processing device 10. Also, in Figure 4, the display 12 is installed on a horizontal plane, and the Xp axis and Xs axis, the Yp axis and Ys axis, and the Zp axis and Zs axis coincide.

[0045] When the display 12 is mounted on the front of the information processing device 10, the user usually operates the information processing device 10 from the front, so it can be said that the user is often in front of the information processing device 10. Therefore, in this case, the direction in front of the display 12 (in front of the information processing device 10) is the direction in which the user is presumed to be located. Accordingly, as shown in Figure 4, a predetermined area at the rear of the spatial area 30 can be set as a non-detection area 30a (shaded area).

[0046] Figure 5 is a perspective view of the display 12. As shown in Figure 5, consider the case where the display 12 is erected, or the case where the display 12 is tilted so that it faces upward in the vertical direction (positive Zs axis direction) and horizontally (positive Ys axis direction in Figure 5). In this case, the direction of extension of the user's arm from the tip of the operating body 32 (the user's fingertip or the tip of the stylus held by the user) toward the user's torso is often downward (positive Yp axis direction) along the plane parallel to the display 12 (direction of the XpYp plane). Therefore, in this case, the downward direction (positive Yp axis direction) along the plane parallel to the display 12 is the direction in which the user is presumed to be located. Therefore, as shown in Figure 5, when the position of part 32a is detected as the first position within the spatial region 30, a predetermined area in the upward direction (negative Yp axis direction) along the plane parallel to the display 12 within the spatial region 30 can be set as the undetected area 30a.

[0047] Figure 6 is a plan view of the display 12. Consider the case where characters are displayed on the display 12, as shown in Figure 6. In this case, the direction of extension of the user's arm from the tip of the operating body 32 toward the user's torso is often opposite to the upright direction of the characters (here, the direction from the bottom to the top of the upright characters). Therefore, in this case, the direction opposite to the upright direction of the characters is the direction in which the user is presumed to be located. Accordingly, as shown in Figure 6, a predetermined area within the spatial region 30 that is on the side of the upright direction of the characters relative to the position of part 32a can be set as a non-detection area 30a.

[0048] The information processing device 10 may be a table-shaped device, with a display 12 provided on the table surface, and may be a device that allows non-contact operation of the display 12 from multiple directions. Figure 7 is a plan view of the display 12 of such an information processing device 10. In such a display 12, the edge of the display 12 is in the direction where the user is presumed to be located.

[0049] For example, suppose the display 12 is assumed to be viewable from two directions, as shown in Figure 7: the positive Yp-axis side (bottom of Figure 7) and the negative Yp-axis side (top of Figure 7). In this case, the positive Yp-axis side and the negative Yp-axis side of the display 12 are the directions in which the user is presumed to be located. Therefore, the input determination unit 22 can set a predetermined area on the center side of the spatial area 30 in the Yp-axis direction as a non-detection area 30a. If the display 12 is assumed to be viewable from four directions, including the positive Xp-axis side (right side of Figure 7) and the negative Xp-axis side (left side of Figure 7), the input determination unit 22 can set a predetermined area on the center side of the spatial area 30 in the Xp-axis direction and on the center side of the spatial area 30 in the Yp-axis direction as a non-detection area 30a.

[0050] Furthermore, if the information processing device 10 has a sensor (for example, a camera that takes pictures of the area around the information processing device 10) that detects a user in its vicinity, the input determination unit 22 may acquire the direction in which the user is located based on the detection signal from the sensor and set the non-detection area 30a accordingly. Also, if the direction in which the user is located changes dynamically, the non-detection area 30a may be changed dynamically accordingly.

[0051] Furthermore, the input determination unit 22 can designate the area of ​​the spatial region 30 opposite to the screen region containing the operation icon or message as a non-detection region 30a.

[0052] For example, as shown in Figure 8, when an operation icon 34 that is the target of contactless operation is displayed on the display 12, the area opposite the screen area containing the operation icon 34 can be set as a non-detection area 30a. As shown in Figure 8, when multiple operation icons 34 are displayed, multiple non-detection areas 30a may be set in the spatial area 30. Note that the non-detection area 30a can be set for each screen.

[0053] Incidentally, while a user is performing a non-contact operation on the display 12 using the operating body 32, there is a possibility that, contrary to the user's intention, the object detection unit 20 may detect a part of the operating body 32 other than the tip as the nearest point of contact. For example, as shown in Figure 2, when a user is performing a non-contact operation using the finger, which is the operating body 32, at the start of the non-contact operation, the object detection unit 20 detects the tip of the index finger, part 32a, as the nearest point of contact, as intended by the user. However, if the user changes the angle of their arm, and the operating body 32 takes on the posture shown in Figure 9, the distance Lvd in the Zp axis direction between part 32d, which is the wrist, and the display 12 becomes smaller than the distance Lva in the Zp axis direction between part 32a and the display 12, and part 32d may be detected as the nearest point of contact. In this case, the input position becomes the position D corresponding to the nearest point of contact, part 32d, rather than the position A intended by the user.

[0054] Thus, when a user is performing a non-contact operation on the display 12 using the operating body 32, and a part other than the tip of the operating body 32 is detected as the nearest point of contact against the user's intention, the position of the nearest point of contact changes significantly in a very short time. Here, a very short time refers to a very short period of time, such as tens of milliseconds or less. For example, in the example in Figure 9, when the user is performing a non-contact operation with the index finger as the operating body 32, it is conceivable that the tip of the index finger, part 32a, may move slightly due to the user's operation, but it is unlikely that the position of part 32a would move to the position of the wrist, part 32d, in a very short time.

[0055] Therefore, if the distance in the plane parallel to the display 12 between the first position of the nearest point detected at the first time point and the second position of the nearest point detected at the second time point, a small amount of time after the first time point, is greater than or equal to the second threshold distance, the input determination unit 22 should consider the position of the display 12 corresponding to the second position as the instruction input position as an incorrect determination. In other words, the input determination unit 22 should not consider the position of the display 12 corresponding to the second position as the instruction input position.

[0056] This will be explained in detail with reference to Figure 9. As described above, the input determination unit 22 detects the nearest point at a predetermined movement determination time interval. Here, the movement determination time is assumed to be a very short time (preferably several tens of milliseconds or less).

[0057] Suppose that at the first time point, the object detection unit 20 determines that part 32a, which is the tip of the index finger, is the nearest point of contact. At this time, the input determination unit 22 stores information indicating the position of part 32a, in particular the Xp and Yp coordinates of part 32a. Subsequently, suppose the user changes the posture of the operating body 32 as shown in Figure 9, and at the second time point, after the movement determination time has elapsed from the first time point, the object detection unit 20 determines that part 32d, which is the wrist, is the nearest point of contact. At this time, the input determination unit 22 obtains the Xp and Yp coordinates of part 32d and calculates the distance between the position of part 32a, which is the nearest point of contact at the first time point, and part 32d, which is the nearest point of contact at the second time point, in a plane parallel to the display 12 (i.e., the XpYp plane direction).

[0058] In this embodiment, the input determination unit 22 calculates the distance Lh between position A on the display 12 corresponding to the position of part 32a, which is the nearest part at the first time point, and position D on the display 12 corresponding to the position of part 32d, which is the nearest part at the second time point. In other words, the input determination unit 22 calculates the distance between the instruction input position at the first time point and the instruction input position at the second time point. Note that the distance between part 32a and part 32d in the plane parallel to the display 12 does not necessarily have to be calculated using the distance between position A and position D on the display 12; the distance between part 32a and part 32d in the XpYp plane direction in the spatial region 30 may be calculated directly.

[0059] The input determination unit 22 then compares the calculated distance Lh with a predetermined second threshold distance. If distance Lh is greater than or equal to the second threshold distance, the determination that position D is the input position is incorrect. If distance Lh is less than the second threshold distance, the input determination unit 22 determines that position D is the input position.

[0060] If the input determination unit 22 incorrectly determines that the position of the display 12 corresponding to the second position is the instruction input position, it can invalidate the instruction input for that instruction input position. Alternatively, if the input determination unit 22 incorrectly determines that the position of the display 12 corresponding to the second position is the instruction input position, the notification processing unit 26 (details described later) can output a warning to the user.

[0061] As described above, the input determination unit 22 can perform the following processes: if it continues to detect the nearest neighbor for a threshold time within a local area in the spatial area 30, it can perform the following processes: if the distance in the plane parallel to the display 12 between the first position of the nearest neighbor detected at the first time point and the second position of the nearest neighbor detected at the second time point after a small amount of time has elapsed from the first time point is greater than or equal to the second threshold distance it continues to detect the nearest neighbor for a threshold time within a local area within the spatial area 30, it can perform the following processes: if it continues to detect the nearest neighbor for a threshold time within a local area within the spatial area 30, it can perform the following processes: if it continues to detect the nearest neighbor for a threshold time within a local area within the spatial area 30, it can perform the following processes: if the distance in the plane parallel to the display 12 between the first position of the nearest neighbor detected at the first time point and the second position of the nearest neighbor detected at the second time point after a small amount of time has elapsed from the first time point is greater than or equal to the second threshold distance, it can perform the following processes: if it continues to detect the nearest nearest neighbor for a threshold time within a local area within the spatial area 30, it can perform the following processes: if it continues to detect the nearest nearest neighbor for a threshold time within a local area within the spatial area 30, it can perform the following processes: if it continues to detect the nearest nearest neighbor for a threshold time within a local area within the spatial area 30, it can perform the following processes: if the distance in the plane parallel to the display 12 corresponding to the position of the nearest neighbor detected at the first time point and the second position of the nearest nearest neighbor detected at the second time point

[0062] Here, the input determination unit 22 may select and execute an appropriate process from the first misjudgment process and the second misjudgment process depending on the distance in the plane parallel to the display 12 between the first position, which is the position of the nearest point at the first time point, and the second position, which is the position of the nearest point at the second time point. Specifically, the input determination unit 22 may make a misjudgment if the distance in the plane parallel to the display 12 between the first position and the second position is less than the second threshold distance, and the nearest point has been continuously detected within a local area in the spatial area 30 for a threshold time, and the determination that the position on the display 12 corresponding to the position of the nearest point is the instruction input position is misjudgment if the distance in the plane parallel to the display 12 between the first position of the nearest point detected at the first time point and the second position of the nearest point detected at the second time point, which is a small time elapsed from the first time point, is greater than or equal to the second threshold distance, and the determination that the position on the display 12 corresponding to the second position is the instruction input position is misjudgment.

[0063] Let's explain this in detail with reference to Figure 10. Figure 10 is a diagram showing the threshold distance and the second threshold distance as viewed from the Zp axis direction. Point F represents the position of the nearest neighbor at the first time point, i.e., the first position. D1 is the threshold distance referenced in the first misclassification process, and the inner circle represents the position at a threshold distance D1 away from the first position F. D2 is the second threshold distance referenced in the second misclassification process, and the outer circle represents the position at a second threshold distance D2 away from the first position F.

[0064] Assume the second position is the position indicated by the symbol S1 in Figure 10. In this case, the distance between the first position F and the second position S1 in the plane parallel to the display 12 is less than the second threshold distance D2 and less than the threshold distance D1, so the input determination unit 22 starts the first misjudgment process. That is, if the input determination unit 22 determines that it has continued to detect the nearest neighbor for a threshold time within a local area in the spatial area 30, it considers the determination to set the position on the display 12 corresponding to the position of the nearest neighbor as the instruction input position to be a misjudgment. As described above, in this embodiment, if the distance between the first position F and the second position S1 in the plane parallel to the display 12 is less than the threshold distance D1, the input determination unit 22 then determines whether the distance between the second position S1 and the position of the nearest neighbor at the third time point after the movement determination time from the second time point is less than the threshold distance D1. This process is repeated at each movement determination time, and if it is determined that the nearest neighbor has been continuously detected within a local area within the spatial area 30 for a threshold time, the determination to set the position on the display 12 corresponding to the position of the nearest neighbor as the instruction input position is considered an incorrect determination.

[0065] Assume that the position of the nearest point at the second time point, i.e., the second position, is the position indicated by symbol S2 in Figure 10. In this case, since the distance between the first position F and the second position S2 in the plane parallel to the display 12 (XpYp plane) is greater than or equal to the second threshold distance D2, the input determination unit 22, through the second misjudgment process, determines that the position on the display 12 corresponding to the second position S2 is the instruction input position to be a misjudgment.

[0066] If the second position is the position indicated by the symbol S3 in Figure 10, that is, if the distance between the first position F and the second position S3 in a plane parallel to the display 12 is less than the second threshold distance D2 and greater than or equal to the threshold distance D1, the input determination unit 22 does not consider the determination that the position on the display 12 corresponding to the position of the second position S3 is the instruction input position to be an incorrect determination, and makes the instruction input position valid.

[0067] Returning to Figure 1, the threshold time setting unit 24 sets the threshold time as described above. Within the spatial region 30, there may be areas where parts of objects unintended by the user tend to remain. For example, in an information processing device 10 with a display 12 positioned at the front, as shown in Figure 4, parts of objects unintended by the user tend to remain more often on the front side of the spatial region 30 (positive direction of the Yp axis) compared with the back side of the spatial region 30 (negative direction of the Yp axis). In areas where parts of objects unintended by the user tend to remain, a shorter threshold time is preferable in order to make an error in determining the position on the display 12 corresponding to the position of the part (nearest part) as the input position earlier. On the other hand, in areas where parts of objects unintended by the user are less likely to remain, the threshold time does not need to be shorter than in areas where parts of objects unintended by the user tend to remain, in order to improve the accuracy of error detection.

[0068] First, the input determination unit 22 continuously detects the nearest neighbor within a local area in the spatial area 30 for a threshold time, and if it incorrectly determines that the position on the display 12 corresponding to the position of the nearest neighbor is the instruction input position, it stores the incorrect determination position, which is the position of the nearest neighbor at the time of the incorrect determination, in the memory 16.

[0069] As shown in Figure 11, the threshold time setting unit 24 divides the spatial region 30 into multiple sub-spatial regions 36 and counts the number of times a misjudgment location is detected within each of the multiple sub-spatial regions 36. The threshold time setting unit 24 then sets the threshold time for each of the multiple sub-spatial regions 36 such that the threshold time becomes shorter the more times a misjudgment location is detected, or in other words, the threshold time becomes longer the fewer times a misjudgment location is detected. In the example in Figure 11, the spatial region 30 is divided into four sub-spatial regions 36, but of course, the spatial region 30 may be divided into any other number of sub-spatial regions 36.

[0070] Then, when an object enters the spatial region 30, the input determination unit 22 identifies a sub-spatial region 36 where the detected nearest neighbor is located, based on the position of the nearest neighbor. If the nearest neighbor is detected for a threshold time set for the sub-spatial region 36 within a local area within the identified sub-spatial region 36, the determination that the position on the display 12 corresponding to the position of the nearest neighbor is the instruction input position is incorrect.

[0071] As described above, the threshold time set for each subspace region 36 may be a parameter common to all users, but the threshold time setting unit 24 may set the threshold time for each subspace region 36 for each user. In this case, the input determination unit 22 continuously detects the threshold time and nearest location within a local region of the space region 30, and if it incorrectly determines that the position on the display 12 corresponding to the position of the nearest location is the instruction input position, it stores the incorrect determination position, which is the position of the nearest location at the time of the incorrect determination, in the memory 16 for each user. For example, the input determination unit 22 stores the user ID that identifies the user and the incorrect determination position in association with each other in the memory 16. The user ID can be obtained by the processor 18 authenticating the user prior to contactless operation on the display 12.

[0072] Then, the threshold time setting unit 24 counts for each of the multiple subspace regions 36 the number of times a misjudgment location is detected within that subspace region 36 for each user, and sets the threshold time for each of the multiple subspace regions 36 for each user such that the threshold time becomes shorter the more times a misjudgment location is detected.

[0073] Of course, in this case, when an object enters the spatial region 30, the input determination unit 22 identifies the user ID of the user currently logged into the information processing device 10, identifies a sub-spatial region 36 where the detected nearest nearest object is located based on the position of the nearest nearest object, and if the nearest nearest object is detected within a local area within the identified sub-spatial region 36 for a threshold time set for the user and the sub-spatial region 36, the determination that the position on the display 12 corresponding to the position of the nearest nearest object should be the instruction input position is incorrect.

[0074] The notification processing unit 26 performs a process to notify the user of the judgment result of the input judgment unit 22. For example, the notification processing unit 26 displays a pointer on the display 12 that indicates the instruction input position determined by the input judgment unit 22.

[0075] Furthermore, if the input determination unit 22 incorrectly determines that the position on the display 12 corresponding to the nearest point is the instruction input position, the notification processing unit 26 outputs a warning to the user. When the notification processing unit 26 outputs a warning to the user, the input determination unit 22 may or may not invalidate the instruction input for the position on the display 12 corresponding to the nearest point.

[0076] The warning may be one that notifies the user that there has been a misjudgment of the instruction input position. For example, the notification processing unit 26 displays a message on the display 12 indicating that there has been a misjudgment of the instruction input position. In addition to displaying the warning on the display 12, the notification processing unit 26 may also output the warning by means of sound or vibration, or by a combination of at least two of these methods. Furthermore, the warning may be one that notifies the user of the position on the display 12 corresponding to the position of the nearest touch point. For example, the notification processing unit 26 displays a pointer with an unusual display pattern (e.g., a different color or blinking) at the position on the display 12 corresponding to the position of the nearest touch point. This allows the user to understand the position of the nearest touch point that was misjudged.

[0077] The configuration of the information processing device 10 according to this embodiment is as described above. The processing flow of the information processing device 10 according to this embodiment will now be explained according to the flowchart shown in Figure 12.

[0078] In step S10, the object detection unit 20 determines, based on the detection signal from the object sensor 14, whether or not it has detected an object in the spatial area 30 facing the display 12, which is the operating surface to be operated without contact. Here, assuming that an object has been detected in the spatial area 30, the process proceeds to step S12.

[0079] In step S12, the object detection unit 20 detects the nearest part of the object in the spatial region 30 that is the part that is closest to the display 12 in the direction perpendicular to the display 12 (Zp axis direction), and obtains the current position, which is the location of that part.

[0080] In step S14, the input determination unit 22 determines whether the previous position, which is the position of the nearest contact point detected at the previous point, is retained. Here, it is determined that the previous position is not retained, and the process proceeds to step S16.

[0081] In step S16, the input determination unit 22 stores the current position of the nearest point obtained in step S12 as the previous position in the memory 16. Here, the previous position becomes the first position. The notification processing unit 26 also displays a pointer on the display 12 at the position corresponding to the previous position.

[0082] In step S18, the input determination unit 22 waits for a small amount of time, which is the movement determination time.

[0083] In step S10 again, the object detection unit 20 determines whether or not it has detected an object in the spatial region 30. If no object is detected, that is, if the object including the nearest point detected in step S12 has moved outside the spatial region 30, the process proceeds to step S20. In step S20, the input determination unit 22 erases the previous position held in step S16 from the memory 16. The notification processing unit 26 also hides the pointer that was displayed in step S16. In this case, assuming that an object has been detected in the spatial region 30 again in step S10, the process proceeds to step S12.

[0084] In step S12 again, the object detection unit 20 detects the nearest point again and obtains its current position. Here, the current position becomes the second position.

[0085] In step S14 again, the previous position is retained, so the process proceeds to step S21.

[0086] In step S21, the input determination unit 22 determines whether the current position is within the non-detection area 30a. If the current position is within the non-detection area 30a, the process proceeds to step S16; otherwise, the process proceeds to step S22.

[0087] In step S22, the input determination unit 22 determines whether the distance between the previous position (in this case, the first position) and the current position (in this case, the second position) in a plane parallel to the display 12 is greater than or equal to the second threshold distance D2. If the distance is greater than or equal to the second threshold distance D2, the process proceeds to step S24.

[0088] In step S24, the input determination unit 22 determines that the determination to set the position on the display 12 corresponding to the current position as the instruction input position is incorrect. Then, the input determination unit 22 invalidates the instruction input for the position on the display 12 corresponding to the current position, or the notification processing unit 26 outputs a warning to the user.

[0089] In step S22, if the distance between the previous position and the current position in a plane parallel to the display 12 is less than the second threshold distance D2, the process proceeds to step S26.

[0090] In step S26, the input determination unit 22 determines whether the distance between the previous position and the current position in a plane parallel to the display 12 is less than the threshold distance D1. If the distance is not less than the threshold distance D1, the process proceeds to step S28. In step S28, the input determination unit 22 clears the counter (a counter used to determine that the nearest neighbor has been continuously detected for a threshold time within a local area in the spatial region 30) and returns to step S16.

[0091] In step S26, the input determination unit 22 determines that the distance between the previous position and the current position in a plane parallel to the display 12 is less than the threshold distance D1, and proceeds to step S30. In step S30, the input determination unit 22 increments the counter by 1. This counter is cleared at the start of this flowchart.

[0092] In step S32, the input determination unit 22 determines whether the counter has reached a predetermined value N. If the counter has reached the predetermined value N, the input determination unit 22 determines that it has continuously detected the nearest neighbor for a threshold time within the local region of the spatial region 30, and proceeds to step S24. If the counter has not reached the predetermined value N, the process returns to step S16, and steps S16, S18, S10, S12, S14, S22, S26, and S30 are repeated.

[0093] Although embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications are possible without departing from the spirit of the invention. [Explanation of Symbols]

[0094] 10 Information processing device, 12 Display, 14 Object sensor, 16 Memory, 18 Processor, 20 Object detection unit, 22 Input determination unit, 24 Threshold time setting unit, 26 Notification processing unit, 30 Spatial area, 30a Non-detection area, 32 Operation body, 32a, 32b, 32c, 32d Parts, 34 Operation icon.

Claims

1. Equipped with a processor, The aforementioned processor, The nearest point is detected, which is the part of an object in the spatial region opposite the operating surface that is the closest to the operating surface in the direction perpendicular to the operating surface. The position on the operating surface corresponding to the position of the nearest contact point is determined as the instruction input position by the user's non-contact operation. If the nearest neighbor is continuously detected within a local area within the aforementioned spatial region for a threshold time, the instruction input for the position on the operating surface corresponding to the position of the nearest neighbor is disabled. An information processing device characterized by the following:

2. Equipped with a processor, The aforementioned processor, The nearest point is detected, which is the part of an object in the spatial region opposite the operating surface that is the closest to the operating surface in the direction perpendicular to the operating surface. The position on the operating surface corresponding to the position of the nearest contact point is determined as the instruction input position by the user's non-contact operation. If the nearest neighbor is continuously detected within a local area of ​​the spatial region for a threshold time, a warning is output to the user. An information processing device characterized by the following:

3. The aforementioned processor, If the nearest neighbor is continuously detected within a local area within the spatial region for a threshold time, the determination that the position on the operating surface corresponding to the position of the nearest neighbor is the instruction input position is considered an incorrect determination. The information processing apparatus according to claim 1 or 2.

4. The aforementioned processor, If the nearest neighbor is continuously detected for a threshold time within the local area which is part of the spatial area which is a predetermined non-detection area, the determination that the position on the operating surface corresponding to the position of the nearest neighbor is the instruction input position will not be considered a false determination. The information processing apparatus according to claim 3.

5. The aforementioned non-detection area is the area located on the opposite side from the direction in which the user is presumed to be located. The information processing apparatus according to feature 4.

6. If the operating surface is erected vertically, or if the operating surface is inclined to face vertically upward and horizontally, the downward direction along the planar direction parallel to the operating surface is the direction in which the user is presumed to be located. The information processing apparatus according to feature 4.

7. The aforementioned processor, The misjudgment position, which is the position of the nearest point at the time of the misjudgment, is stored in memory. For each of the multiple subspace regions within the aforementioned spatial region, the threshold time is set such that the threshold time decreases as the number of times the misjudgment position is detected within that subspace region increases. If, within the local region within the subspace region, the nearest neighbor is continuously detected for the threshold time set for the subspace region, then the determination that the position on the operating surface corresponding to the position of the nearest neighbor is the instruction input position is incorrect. The information processing apparatus according to claim 3.

8. The aforementioned processor, The aforementioned misjudgment locations are stored in memory for each user. The threshold time for each of the aforementioned subspace regions is set for each user. The information processing apparatus according to feature 7.

9. The aforementioned processor, If the distance in the planar direction parallel to the operating surface between the first position, which is the position of the nearest point detected at the first time point, and the second position, which is the position of the nearest point detected at the second time point, a small time after the first time point, is less than the threshold distance, and if the nearest point continues to be detected within the local region for a threshold time, then the determination that the position on the operating surface corresponding to the position of the nearest point is the instruction input position is incorrect. If the distance between the first position and the second position in a planar direction parallel to the operating surface is greater than or equal to the threshold distance, the determination that the position on the operating surface corresponding to the second position is the instruction input position is incorrect. The information processing apparatus according to claim 3.

10. On the computer, The nearest point is detected, which is the part of an object in the spatial region opposite the operating surface that is the closest to the operating surface in the direction perpendicular to the operating surface. The position on the operating surface corresponding to the position of the nearest contact point is determined as the instruction input position by the user's non-contact operation. If the nearest neighbor is continuously detected within a local area within the aforementioned spatial region for a threshold time, the instruction input for the position on the operating surface corresponding to the position of the nearest neighbor is disabled. An information processing program characterized by the following features.

11. On the computer, The nearest point is detected, which is the part of an object in the spatial region opposite the operating surface that is the closest to the operating surface in the direction perpendicular to the operating surface. The position on the operating surface corresponding to the position of the nearest contact point is determined as the instruction input position by the user's non-contact operation. If the nearest neighbor is continuously detected within a local area of ​​the spatial region for a threshold time, a warning is output to the user. An information processing program characterized by the following features.

Images