Virtual keypad motion recognition device

Abstract

Disclosed is a virtual keypad motion recognition device. The virtual keypad motion recognition device comprises: a processor and a memory operatively connected to the processor. The memory stores instructions that, when executed, instruct the processor to: receive position information obtained by tracking the motion of at least one finger of a user relative to the virtual keypad output through an output means, the position information including position information about a first axis and position information about a second axis; identifying the position information at a certain time interval, wherein a first point in time, a second point in time, and a third point in time, which are successive points in time spaced apart by the certain time interval, are identified, and a selection point in time, at which a valid region of the virtual keypad is estimated to have been selected, and whether the selection made by the user is valid are determined on the basis of position information about the first axis at the first point in time, position information about the first axis at the second point in time, and position information about the first axis at the third point in time.

Description

Virtual keypad gesture recognition device

[0001] The present invention relates to a virtual keypad motion recognition device, and more particularly, to a device that recognizes a user's motion on a virtual keypad and identifies a selection area.

[0002]

[0003] The content described in this section merely provides background information for the present embodiment and does not constitute prior art.

[0004] Recently, there has been a growing demand for technologies that replace physical input interfaces, such as keyboards or keypads. For example, technologies are being used to project keypad-shaped beams and analyze user movements using sensors to provide a virtual interface that replaces physical input interfaces. Alternatively, technologies are being used to project keypad shapes onto touch panels and analyze the location or area selected by the user, providing a virtual interface that replaces physical input interfaces.

[0005]

[0006] The purpose of the present invention is to provide a virtual keypad motion recognition device for recognizing a user's motion using a virtual keypad.

[0007] In addition, an object of the present invention is to provide a virtual keypad motion recognition device for determining the validity of a user's motion using a virtual keypad.

[0008] The objectives of the present invention are not limited to those mentioned above. Other objectives and advantages of the present invention not mentioned above can be understood through the following description and will be more clearly understood through the embodiments of the present invention. Furthermore, it will be readily apparent that the objectives and advantages of the present invention can be realized by the means and combinations thereof set forth in the claims.

[0009]

[0010] A motion recognition device for a virtual keypad according to an embodiment of the present invention includes a processor and a memory operatively connected to the processor, wherein the memory stores instructions that, when executed, cause the processor to receive position information that tracks movement of at least one finger of a user with respect to the virtual keypad output through an output means, the position information including position information for a first axis and position information for a second axis, identify the position information at regular time intervals, and identify a first time point, a second time point, and a third time point that are consecutively spaced from each other by the regular time intervals, and determine a selection time point at which a valid area of ​​the virtual keypad is presumed to have been selected based on the position information for the first axis at the first time point, the position information for the first axis at the second time point, and the position information for the first axis at the third time point, determine whether the selection by the user is valid at the selection time point, and identify the valid area corresponding to the position information at the selection time point as a selection area based on the determination that the selection by the user is valid.

[0011] In addition, the instructions cause the processor to receive first location information of a first finger and second location information of a second finger as the location information that tracks the movement of the at least one finger, the first location information including location information of the first finger at the first time point, location information of the first finger at the second time point, and location information of the first finger at the third time point, and the second location information including location information of the second finger at the first time point, location information of the second finger at the second time point, and location information of the second finger at the third time point.

[0012] Additionally, the instructions cause the processor to compare positional information about the first axis of each of the first finger and the second finger at each of the first time point, the second time point, and the third time point, and identify a finger among the first finger and the second finger that is presumed to have selected the valid area.

[0013] Additionally, the instructions identify the first finger, which is assumed to have selected the valid area among the first finger and the second finger, by causing the processor to compare the position information about the first axis of the first finger with the position information about the first axis of the second finger at each of the first time point, the second time point, and the third time point and to identify a smaller value.

[0014] Additionally, the instructions cause the processor to compare an average value of a first value, which is position information about the first axis of the first finger at the first time point, and a second value, which is position information about the first axis of the first finger at the third time point, with a value obtained by adding a third value, which is position information about the first axis of the first finger at the second time point, and a set value, to identify the selection time point.

[0015] Additionally, the instructions cause the processor to compare the second value with a value that is the sum of the third value and the set value, to determine whether the user's selection is valid at the time of selection.

[0016] Additionally, the instructions cause the processor to determine that the user's selection of the valid area at the second time is valid based on the average value being greater than the summed value and the summed value being less than the second value.

[0017] Additionally, the instructions cause the processor to determine that the user's selection of the valid area at the second time point is invalid based on at least one of the average value being less than the sum value or the sum value being greater than the second value.

[0018] Additionally, the instructions cause the processor to receive a photographed image obtained by photographing the virtual keypad and the user's hand using a photographing means, the photographed image including the first axis and the second axis.

[0019] A virtual keypad operation recognition method according to an embodiment of the present invention includes the steps of receiving position information including position information for a first axis and position information for a second axis as position information that tracks movement of at least one finger of a user with respect to the virtual keypad output through an output means, the step of identifying the position information that tracks the movement of the at least one finger at regular time intervals, the step of determining, based on the position information for the first axis, a selection point at which a valid area of ​​the virtual keypad is presumed to have been selected for a first time point, a second time point, and a third time point having the regular time intervals from each other, a step of determining whether the selection by the user is valid at the selection point, and a step of identifying the valid area corresponding to the position information at the selection point as a selection area based on the determination that the selection by the user is valid.

[0020]

[0021] The virtual keypad motion recognition device of the present invention can improve the accuracy of selection area identification by recognizing a user's motion on the virtual keypad based on location information for consecutive points in time at a certain time interval.

[0022] In addition, the virtual keypad motion recognition device of the present invention can improve the accuracy of determining the validity of a selection motion by identifying the selection point and the validity of the selection with respect to the user's motion on the virtual keypad.

[0023] In addition to the above-described contents, the specific effects of the present invention are described together with the specific matters for carrying out the invention below.

[0024]

[0025] FIG. 1 is a drawing for explaining a virtual keypad motion recognition device according to an embodiment of the present invention.

[0026] Figure 2 is a drawing for explaining the processor of Figure 1.

[0027] Figure 3 is a flowchart for explaining the operation of the processor of Figure 1.

[0028] FIG. 4a, FIG. 4b, FIG. 4c, and FIG. 4d are drawings for explaining step S100 of FIG. 3.

[0029] FIG. 5 and FIG. 6 are drawings for explaining step S200 of FIG. 3.

[0030] FIG. 7, FIG. 8a, FIG. 8b, FIG. 8c and FIG. 9 are drawings for explaining step S300 and step S400 of FIG. 3.

[0031]

[0032] The terms and words used in this specification and claims should not be interpreted based on their general or dictionary meanings. In accordance with the principle that inventors can define the concepts of terms and words to best describe their inventions, they should be interpreted in a way that is consistent with the technical concept of the present invention. Furthermore, the embodiments described in this specification and the configurations depicted in the drawings are merely examples of how the present invention can be realized and do not fully represent the technical concept of the present invention. Therefore, it should be understood that various equivalents, modifications, and applicable examples may exist as of the time of filing.

[0033] The terms first, second, A, B, etc. used in this specification and claims may be used to describe various components, but the components should not be limited by these terms. These terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term "and / or" includes any combination of a plurality of related listed items or any item among a plurality of related listed items.

[0034] The terminology used in this specification and claims is for the purpose of describing specific embodiments only and is not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. It should be understood that terms such as "comprise" or "have" in this application do not preclude the presence or addition of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification.

[0035] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

[0036] Terms defined in commonly used dictionaries should be interpreted as having meanings consistent with their meanings within the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense unless explicitly defined herein. Furthermore, each component, process, procedure, or method included in each embodiment of the present invention may be shared within the scope of non-contradictory technical aspects.

[0037] Hereinafter, a virtual keypad motion recognition device according to an embodiment of the present invention will be described with reference to drawings.

[0038]

[0039] FIG. 1 is a diagram illustrating a virtual keypad motion recognition device according to an embodiment of the present invention. FIG. 2 is a diagram illustrating the processor of FIG. 1. FIG. 3 is a flowchart illustrating the operation of the processor of FIG. 1.

[0040] Referring to FIGS. 1, 2, and 3, a virtual keypad motion recognition device (100) according to an embodiment of the present invention may include a processor (110) and a memory (120).

[0041] One or more other components (e.g., a communication module) may be added to the virtual keypad gesture recognition device (100).

[0042] The memory (120) can store various data used by at least one component (e.g., processor (110)) of the virtual keypad gesture recognition device (100). The data can include, for example, software (e.g., program) and input data or output data for commands related thereto. The memory (120) can include volatile memory or non-volatile memory.

[0043] The memory (120) may store commands, information, or data related to the operations of components included in the virtual keypad motion recognition device (100). For example, the memory (120) may store instructions that, when executed, enable the processor (110) to perform various operations described in this document.

[0044] The processor (110) may be operatively coupled to the memory (120) to perform the overall function of the virtual keypad gesture recognition device (100). The processor (110) may include, for example, one or more processors. The one or more processors may include, for example, an image signal processor (ISP), an application processor (AP), or a communication processor (CP).

[0045] The processor (110) may, for example, execute software (e.g., a program) to control at least one other component (e.g., a hardware or software component) of the virtual keypad gesture recognition device (100) connected to the processor (110) and perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the processor (110) may load a command or data received from another component (e.g., a communication module ()) into the memory (120), process the command or data stored in the memory (120), and store the resulting data in the memory (120). According to one embodiment, the processor (110) may include a main processor (e.g., a central processing unit or an application processor) and an auxiliary processor (e.g., a graphics processing unit, an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together therewith. Additionally or alternatively, the auxiliary processor may be configured to use lower power than the main processor or to be specialized for a given function. The auxiliary processor may be implemented separately from the main processor or as part of it. The program may be stored as software in memory (120) and may include, for example, an operating system, middleware, or an application.

[0046]

[0047] The processor (110) may include a recording unit (111), a calculation unit (113), a judgment unit (115), and a storage unit (117). The recording unit (111) may receive location information from an external source. The recording unit (111) may transmit the received location information to the calculation unit (113). The calculation unit (113) may identify a finger that is assumed to have selected an effective area of ​​a virtual keypad, the time of selection, and the validity of the selection based on the location information. The judgment unit (115) may identify a selection area of ​​a virtual keypad based on the identification result of the calculation unit (113). The storage unit (117) may store at least one of the outputs of the recording unit (111), the calculation unit (113), and the judgment unit (115). The judgment unit (115) may output keypad information. The keypad information may include a finger that is assumed to have selected an effective area of ​​a virtual keypad, the time of selection, the validity of the selection, and the selection area of ​​the virtual keypad.

[0048]

[0049] The virtual keypad may be a physical keypad reproduced on a projection surface through an output means. The projection surface may be, for example, a flat projection surface. The output means may be, for example, a device capable of implementing and outputting an image of the physical keypad. The output means may be, for example, a camera device or a beam projector. When a user makes a motion to select a portion of the virtual keypad using a hand, the user's hand motion may be recognized by a position tracking device. When the user's hand motion is recognized by the position tracking device, position information about the user's finger may be generated. When the user selects a portion of the virtual keypad using a hand, the same input signal as that of the physical keypad may be generated.

[0050]

[0051] The processor (110) (e.g., recording unit (111)) of the virtual keypad motion recognition device (100) can receive location information that tracks the movement of at least one finger of the user on the virtual keypad output through an output means (S100).

[0052] The position information may include position information for a first axis and position information for a second axis. The first axis and the second axis may be axes extending in directions intersecting each other. For example, the first axis may be the y-axis and the second axis may be the x-axis.

[0053] FIG. 4a, FIG. 4b, FIG. 4c, and FIG. 4d are drawings for explaining step S100 of FIG. 3.

[0054] Referring to FIGS. 1, 2, 3, and 4A, a virtual keypad output through an output means may be, for example, a non-dense keypad. A non-dense keypad may be, for example, a keypad including a relatively small number of keys, rather than a dense keypad (or dense key interface) used together with a laptop or a PC. When a specific key among the keys included in the non-dense keypad is selected, a command for executing a specific operation of a specific device by the selected key may be output. A plurality of keys of the non-dense keypad may be arranged to be spaced apart from each other at a certain interval, for example. Accordingly, when a non-dense keypad is output through an output means, the effective areas of the virtual keypad corresponding to the plurality of keys of the non-dense keypad may be output to be spaced apart from each other.

[0055] Referring to FIGS. 1, 2, 3, and 4B, a virtual keypad can be output to a projection surface via an output means (13). The projection surface can be, for example, a flat projection surface. The virtual keypad output via the output means (13) can be captured by a photographing means (11). For example, when a user selects an effective area of ​​the virtual keypad using a finger, this can be captured by the photographing means (11) and generated as a captured image. The captured image can be a trace of the movement of a user's finger using the virtual keypad to obtain location information.

[0056] Referring to FIGS. 1, 2, 3, and 4C, the photographing means (11) of FIG. 4B can generate a photographed image of a user's finger on a virtual keypad. Based on the photographed image by the photographing means (11), the movement of at least one finger of the user can be tracked. The tracking of the finger movement can be tracked by a separate device or software. The tracking result of the movement of at least one finger of the user can be received as location information by the processor (110) (e.g., the recording unit (111)). The processor (110) can further receive, for example, the photographed image.

[0057] Referring to FIGS. 1, 2, 3, and 4D, FIG. 4D illustrates a captured image of a virtual keypad and a user's finger using the virtual keypad captured by a capturing means (11 in FIG. 4B). Based on the captured image, a first axis (y-axis) and a second axis (x-axis) can be determined. Location information, which is a result of tracking the movement of at least one finger of the user, may include location information (e.g., b) for the first axis (y-axis) and location information (e.g., a) for the second axis (x-axis). The processor (110) may receive the location information from an external source. For example, the location information may include first location information of a first finger (f1) and second location information of a second finger (f2) among at least one finger of the user. The captured image of FIG. 4D may represent, for example, location information (e.g., T(a, b)) at a specific point in time.

[0058]

[0059] Referring again to FIGS. 1, 2, and 3, the processor (110) can identify location information at regular time intervals. The processor (110) (e.g., the operation unit (113)) can identify a first, a second, and a third consecutive time points having regular time intervals from each other (S200). The processor (110) can determine a selection finger, a selection time point, and whether the selection is valid for a judgment unit including the first, second, and third consecutive time points having regular time intervals from each other.

[0060] FIG. 5 and FIG. 6 are drawings for explaining step S200 of FIG. 3.

[0061] Referring to FIGS. 1, 2, 3, 4d, 5, and 6, a first time point (T1), a second time point (T2), and a third time point (T3) that are consecutive and have a certain time interval from each other may be one judgment unit. For example, the processor (110) may identify location information at each time point for each of the user's fingers. For example, when FIG. 5 is location information for the first finger, the location information of the first finger may include location information of the first time point (T1(x1, y1)), location information of the second time point (T2(x2, y2)), location information of the third time point (T3(x3, y3)), and location information of the fourth time point (T4(x4, y4)).

[0062] The processor (110) may determine, for example, a selection finger, a selection point, and whether the selection is valid for three consecutive points in time (e.g., a judgment unit). The number of points included in a judgment unit is exemplary and is not limited thereto.

[0063] For example, as described with reference to FIG. 4D, the location information may include first location information of a first finger (f1) and second location information of a second finger (f2) among at least one finger of the user. The first location information may include location information of the first finger (f1) at a first time point (T1) (T1(x11, y11)), location information of the first finger (f1) at a second time point (T2) (T2(x12, y12)), and location information of the first finger (f1) at a third time point (T3) (T3(x13, y13)). The second location information may include location information (T1(x21, y21)) of the second finger (f2) at a first time point (T1), location information (T2(x22, y22)) of the second finger (f2) at a second time point (T2), and location information (T3(x23, y23)) of the second finger (f2) at a third time point (T3).

[0064] The processor (110) (e.g., the operation unit (113)) can identify the finger that is assumed to have selected the valid area of ​​the virtual keypad among the user's first finger (f1) and second finger (f2). In order to identify the finger that is assumed to have selected the valid area of ​​the virtual keypad, the processor (110) can compare the position information for the first axis (e.g., the y-axis) of each of the first finger and the second finger at each of the first time point (T1), the second time point (T2), and the third time point (T3).

[0065] The processor (110) can identify a finger that is assumed to have selected an effective area of ​​the virtual keypad by comparing the position information of the first axis of the first finger with the position information of the first axis of the second finger at each of a first time point (T1), a second time point (T2), and a third time point (T3) and identifying a smaller value. For example, the processor (110) can identify a smaller value by comparing y11 and y21 at the first time point (T1), identify a smaller value by comparing y12 and y22 at the second time point (T2), and identify a smaller value by comparing y13 and y23 at the third time point (T3). For example, if the difference between the first axis of each finger at each of the first time point (T1) and the third time point (T3) is less than or substantially equal to the threshold value, but at the second time point (T2), y12 is less than y22 and the difference is equal to or greater than the threshold value, the processor (110) can identify the first finger as the finger that is assumed to have selected the valid area.

[0066]

[0067] Referring back to FIGS. 1, 2, 3, and 6, the processor (110) (e.g., the operation unit (113)) can determine a selection point at which it is assumed that a valid area of ​​the virtual keypad has been selected, and whether the selection made by the user at the selection point is valid (e.g., validity of the selection) (S300). The processor (110) can determine the selection point and validity of the selection based on position information about the first axis at the first point in time, position information about the first axis at the second point in time, and position information about the first axis at the third point in time.

[0068] The processor (110) can identify an average value of a first value (y11), which is position information about the first axis of the first finger (f1) at a first time point (T1), and a second value (y13), which is position information about the first axis of the first finger (f1) at a third time point (T3). In addition, the processor (110) can identify a value obtained by adding a third value (y12), which is position information about the first axis of the first finger (f1) at a second time point (T2), and a set value (e.g., k). The processor (110) can identify a selection time point by comparing the average value and the summed value.

[0069] The processor (110) (e.g., the judgment unit (115)) can identify a valid area corresponding to the position information at the time of selection as a selection area based on whether the user's selection is valid (S400). If the selection is valid, the processor (110) can correspond the position information for each of the first and second axes at the time of selection with the position information of the plurality of keys included in the virtual keypad. The area where the plurality of keys of the virtual keypad are output on the projection surface may be a valid area. The processor (110) can identify the selection area by comparing the position information for the valid area with the position information at the time of selection. For example, the processor (110) can identify which key of the physical keypad corresponding to the virtual keypad has been selected based on the selection area. Accordingly, a command according to the selected key can be executed.

[0070]

[0071] FIG. 7, FIG. 8a, FIG. 8b, FIG. 8c and FIG. 9 are drawings for explaining step S300 and step S400 of FIG. 3.

[0072] The processor (110) (e.g., the operation unit (113)) can identify the user's selection point. The processor (110) can compare the average value and the summed value (S301).

[0073] The average value may be an average value of a first value at a time point before the reference point and a second value at a time point after the reference point. For example, if the reference point is a second time point (T2), the time point before the reference point may be a first time point (T1), and the time point after the reference point may be a third time point (T3). The first value may be position information of the first axis for one finger of the user. If the first finger (f1) is a finger that is assumed to have touched the valid area of ​​the virtual keypad, the first value may be y11. The second value may be position information of the first axis for one finger of the user. If the first finger (f1) is a finger that is assumed to have touched the valid area of ​​the virtual keypad, the second value may be y13. The processor (110) may identify an average value of the first value and the second value.

[0074] The combined value may be the sum of the third value at the reference point and the set value. For example, the reference point may be the second point in time (T2). The third value may be position information of the first axis for one of the user's fingers. If the first finger (f1) is assumed to be the finger that touched the valid area of ​​the virtual keypad, the third value may be y12. The set value may be a preset value and may be a value for error correction. The processor (110) may identify the sum of the third value and the set value.

[0075] The processor (110) (e.g., the calculation unit (113)) may, if the sum is greater than or equal to the average (NO in S301), set a time point after the reference point as the reference point and then re-judge (S303). For example, if the sum is greater than or equal to the average, the processor (110) may reset the time point after the reference point as the reference point and repeat step S301. For example, if the reference point is the second time point (T2) but the sum is greater than or equal to the average, the reset reference point may be the third time point (T3). If the first finger (f1) is a finger that is assumed to have touched the valid area of ​​the virtual keypad, the processor (110) can reset the reference time to the third time point (T3) and re-determine whether the average value of the first value (y12) at the time point before the reference (the second time point (T2)) and the fourth value (e.g., y14) at the time point after the reference (the fourth time point) is greater than the sum of the third value (y13) at the reference time point (the third time point (T3)) and the set value (e.g., k).

[0076] The processor (110) (e.g., the operation unit (113)) can identify that the reference time is the selection time (S305) when the average value is greater than the sum (YES in S301). For example, when the reference time is the second time (T2), the processor (110) can identify that the user selected the effective area of ​​the virtual keypad at the second time (T2).

[0077] For example, if the user selects the valid area of ​​the virtual keypad using the first finger at the second time point (T2), the first finger will have moved downward along the y-axis direction, which is the first axis, at the second time point (T2) compared to the first time point (T1) and the third time point (T3). In this case, the first time point (T1) is before selection, and after selecting the valid area at the second time point (T2), the first finger will have moved upward along the y-axis direction, which is the first axis, at the third time point (T3).

[0078] The processor (110) (e.g., the operation unit (113)) can determine whether the user's selection action is valid at the time of selection. The processor (110) can compare the second value with the sum of the third value and the set value (S311).

[0079] If the selection point is identified as the second point in time (T2), which is the reference point, the processor (110) can identify a value obtained by adding the third value (y12), which is the position information for the first axis at the second point in time (T2), which is the selection point, and a set value (e.g., k). The processor (110) can compare the added value with the second value (y13), which is the position information for the first axis at the third point in time (T3), which is the point after the reference point.

[0080] The processor (110) (e.g., the calculation unit (113)) may, if the sum is greater than or equal to the second value (NO in S311), set a time point after the reference point as the standard and then re-judge (S313). If the sum is greater than or equal to the second value, this may mean that an action assumed to be selected occurred at the second time point (T2) identified as the selection point, but the selection was invalid. In this case, the processor (110) may re-judge the selection point and the validity of the selection at the selection point. For example, the processor (110) may perform step S303 of FIG. 7 and step S313 of FIG. 8.

[0081] For example, as illustrated in Fig. 8b, at the second time point (T2), which is the reference time point, there was an action assumed to be a selection, but the sum of the third value (y12) and the set value (k) may be greater than or equal to the second value (y13). If the recoil is minimal at the selection time point (T2) and immediately after the selection time point (T3), and the difference (DIF1) between the position values ​​of the first axis of the tip of the first finger (f1) is less than the set value (k), the selection action at the identified selection time point (T2) may be determined to be invalid.

[0082] For example, if the first finger (f1) is a finger that is assumed to have touched the valid area of ​​the virtual keypad, the processor (110) can reset the reference time point to the third time point (T3) and determine whether the third time point (T3) is a selection time point by step S303. For example, if the third time point (T3) is identified as the selection time point by step S303, the processor (110) can compare the sum of the second value (y13), which is position information for the first axis at the third time point (T3), and the set value (e.g., k), with the fourth value (e.g., y14), which is position information for the first axis at the fourth time point (T4), which is a time point after the reference time point.

[0083] The processor (110) (e.g., the operation unit (113)) can determine that the selection at the reference point is a valid selection (S315) if the second value is greater than the sum (YES in S311). For example, if the selection point is the second point in time (T2), which is the reference point in time, the processor (110) can identify the sum of the third value (y12) and the set value (e.g., k). If the sum is less than the second value (y13), which is the position information on the first axis at the third point in time (T3), which is a point in time subsequent to the reference point in time (i.e., the selection point in time), the processor (110) can identify that the selection using the first finger (f1) at the second point in time (T2), which is the selection point in time, is a valid selection.

[0084] For example, as illustrated in Fig. 8c, at the second time point (T2), which is the reference time point, there was an action assumed to be a selection, and the second value (y13) may be greater than the sum of the third value (y12) and the set value (k). If the difference (DIF2) between the position values ​​of the first axis of the tip of the first finger (f1) by the reaction at the selection time point (T2) and immediately after the selection time point (T3) is greater than the set value (k), the selection action at the identified selection time point (T2) may be determined to be valid.

[0085]

[0086] Hereinafter, an example will be described in which the reference time is the second time point (T2) and the user's finger, which is assumed to have selected the valid area of ​​the virtual keypad, is the first finger. When the reference time is the second time point (T2), the average value may be the average value of the first value (y11) of the time point before the reference (the first time point (T1)) and the second value (y13) of the time point after the reference (the third time point (T3)). When the reference time is the second time point (T2), the sum value may be the sum of the third value (y12) of the reference time point (the second time point (T2)) and a set value (e.g., k).

[0087] For example, the processor (110) may determine that the user's selection of the valid area at the second time point (T2) is valid based on the fact that the average value is greater than the sum of the values ​​and that the sum of the values ​​is less than the second value (y13). In this case, the processor (110) may determine that the user has validly selected the valid area corresponding to (x12, y12) at the second time point (T2) using the first finger.

[0088] For example, the processor (110) may determine that the user's selection of the valid area at the second time point (T2) is invalid based on the fact that the average value is less than the sum of the values ​​and that the sum of the values ​​is less than the second value (y13). In this case, the processor (110) may re-evaluate the selection time point and the validity of the selection at the selection time point (e.g., performing steps S303 and S313).

[0089] For example, the processor (110) may determine that the user's selection of the valid area at the second time point (T2) is invalid based on the fact that the average value is greater than the sum of the values ​​and that the sum of the values ​​is greater than the second value (y13). In this case, the processor (110) may re-evaluate the selection time point and the validity of the selection at the selection time point (e.g., performing steps S303 and S313).

[0090] For example, the processor (110) may determine that the user's selection of the valid area at the second time point (T2) is invalid based on the fact that the average value is smaller than the sum and the sum is larger than the second value (y13). In this case, the processor (110) may re-evaluate the selection time point and the validity of the selection at the selection time point (e.g., performing steps S303 and S313).

[0091]

[0092] A virtual keypad motion recognition device (100) according to an embodiment of the present invention can improve the accuracy of motion recognition for a virtual keypad by identifying the time of a user's selection for the virtual keypad and the validity of the selection.

[0093]

[0094] The term "unit" as used herein may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A unit may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a unit may be implemented in the form of an application-specific integrated circuit (ASIC).

[0095] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more commands stored in a storage medium (e.g., a memory (120)) readable by a machine (e.g., a virtual keypad gesture recognition device (100)). For example, a processor (e.g., processor (110)) of a device (e.g., virtual keypad gesture recognition device (100)) can call at least one command among one or more commands stored from a storage medium and execute it. This enables the device to operate to perform at least one function according to the called at least one command. The one or more commands may include code generated by a compiler or code executable by an interpreter. The storage medium readable by the device may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0096] According to one embodiment, the method according to the various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0097] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include a single or multiple entities. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0098]

[0099] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but rather to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

Claims

1. In the motion recognition device of the virtual keypad, processor; and comprising a memory operatively connected to said processor; The above memory, when executed, causes the processor to: Receive position information that tracks the movement of at least one finger of the user on the virtual keypad output through an output means, the position information including position information on a first axis and position information on a second axis, Identifying the above location information at regular time intervals, and identifying a first time point, a second time point, and a third time point that are consecutively spaced apart from each other, Based on the position information about the first axis at the first time point, the position information about the first axis at the second time point, and the position information about the first axis at the third time point, a selection time point at which the valid area of ​​the virtual keypad is presumed to have been selected, and whether the selection by the user at the selection time is valid is determined, Based on the determination that the user's selection is valid, instructions are stored to identify the valid area corresponding to the location information at the time of selection as the selection area. Virtual keypad gesture recognition device.

2. In paragraph 1, The above instructions cause the processor to, Receive first location information of a first finger and second location information of a second finger as the location information that tracks the movement of at least one finger, The first location information includes location information of the first finger at the first time point, location information of the first finger at the second time point, and location information of the first finger at the third time point, The second location information includes location information of the second finger at the first time point, location information of the second finger at the second time point, and location information of the second finger at the third time point. Virtual keypad gesture recognition device.

3. In paragraph 2, The above instructions cause the processor to, By comparing the position information for the first axis of each of the first finger and the second finger at each of the first time point, the second time point, and the third time point, the finger that is presumed to have selected the valid area among the first finger and the second finger is identified. Virtual keypad gesture recognition device.

4. In paragraph 3, The above instructions cause the processor to, At each of the first time point, the second time point, and the third time point, the position information of the first finger with respect to the first axis is compared with the position information of the second finger with respect to the first axis, thereby identifying the smaller value, thereby identifying the first finger that is assumed to have selected the valid area among the first finger and the second finger. Virtual keypad gesture recognition device.

5. In paragraph 4, The above instructions cause the processor to, The first value, which is position information about the first axis of the first finger at the first time point, and the second value, which is position information about the first axis of the first finger at the third time point, are compared with an average value of the third value, which is position information about the first axis of the first finger at the second time point, and a value obtained by adding the set value to identify the selection time point. Virtual keypad gesture recognition device.

6. In paragraph 5, The above instructions cause the processor to, By comparing the sum of the third value and the set value with the second value, it is determined whether the selection by the user at the time of selection is valid. Virtual keypad gesture recognition device.

7. In paragraph 6, The above instructions cause the processor to, Based on the above average value being greater than the above summed value and the above summed value being less than the second value, the user's selection of the valid area at the second time point is determined to be valid. Virtual keypad gesture recognition device.

8. In paragraph 6, The above instructions cause the processor to, Based on at least one of the above average value being less than the summed value or the summed value being greater than the second value, it is determined that the user's selection of the valid area at the second time point is invalid. Virtual keypad gesture recognition device.

9. In paragraph 1, The above instructions cause the processor to, Receive a photographed image obtained by photographing the virtual keypad and the user's hand using a photographing means, The above photographed image includes the first axis and the second axis. Virtual keypad gesture recognition device.

10. In the method of recognizing virtual keypad operation, A step of receiving position information including position information about a first axis and position information about a second axis, wherein the position information is position information that tracks the movement of at least one finger of a user on the virtual keypad output through an output means; A step of identifying the location information that tracks the movement of at least one finger at regular time intervals; A step of determining whether the selection by the user at the selection time is valid, based on the position information for the first axis, for consecutive first time points, second time points, and third time points having a predetermined time interval from each other, a selection time point at which the valid area of ​​the virtual keypad is presumed to have been selected; and A step of identifying the valid area corresponding to the location information at the time of selection as a selection area based on the determination that the user's selection is valid. How to recognize virtual keypad gestures.

Images