METHOD FOR DETECTING A HAND GESTURE

Abstract

A method for recognizing a hand gesture is disclosed. Image data is acquired, wherein the image data represents a two-dimensional image of a user's hand (1). Feature points (2, 3, 4, 5, 6) are determined in the two-dimensional image of the hand (1), wherein the feature points (2, 3, 4, 5, 6) comprise a wrist point (2) and, for each finger of the hand (1), finger joint points (3, 4, 5) and a fingertip point (6), wherein the finger joint points (3, 4, 5) for each finger comprise an MCP joint point (3), a PIP joint point (4), and a DIP joint point (5). A hand position is determined based on the relative positions of the feature points (2, 3, 4, 5, 6) to each other in the two-dimensional image of the hand (1).

Description

[0001] The present invention relates to a method for recognizing a hand gesture. In particular, a hand position can be recognized based on a two-dimensional image of the hand. Hand gestures can be used, in particular, for contactless gesture control of a user interface in a vehicle.

[0002] Modern vehicles typically have at least one control unit that allows the driver or other passengers to control various functions. These can include various vehicle or comfort features, such as navigation system settings, climate control, seat adjustments, lighting settings, and the like. It can also control various functions of an infotainment system, such as playing music, making phone calls, and so on.

[0003] It is common practice in vehicles to control certain functions contactlessly through gestures, particularly hand gestures. For this, a user performs specific predefined gestures, for example, in a designated area of ​​the vehicle cabin above the center console or in front of the dashboard, which are then detected by a corresponding sensor or camera. Gesture control enables intuitive human-machine interaction.

[0004] Other electronic devices, such as computer monitors or smart TVs, can also have user interfaces that allow users to perform various actions, such as selecting channels, changing settings, choosing streaming services, and so on. These types of electronic devices require at least one input device, such as a computer mouse or remote control, which is used to control a cursor and select various actions by clicking or pressing buttons. A touchscreen, operated by touch, may also be used. However, touchless input via gesture control can also be advantageous here, as it eliminates the need for a separate input device.

[0005] Gesture recognition, especially of hand gestures, typically requires three-dimensional scanning of the hand. A 3D sensor, such as a time-of-flight (ToF) camera, can be used for this purpose. This allows for the acquisition of highly accurate data, particularly regarding the hand's position and posture in three-dimensional space, enabling highly accurate hand gesture recognition. However, 3D sensors are relatively expensive, and processing the three-dimensional data requires significant computing power. Complex, trained machine learning models are usually necessary for the analysis.

[0006] Therefore, methods are also known that initially use two-dimensional image data, such as simple camera images. Specific points of the hand can be extracted from these images. However, to recognize hand gestures, the two-dimensional data is often transformed into three-dimensional space. This transformation then allows for the recognition of hand gestures using appropriately trained machine learning models or by comparing them with stored gesture data, as mentioned above. This process is computationally intensive, and the transformations can be prone to errors, especially since two-dimensional image data relating to hand posture can already contain inaccuracies that may be exacerbated by the transformation into three-dimensional data. Therefore, it is often only possible to generate a partially accurate three-dimensional model of the hand from two-dimensional data.

[0007] The present invention aims to simplify the recognition of hand gestures. In particular, the recognition of hand gestures is to be carried out based on two-dimensional image data.

[0008] The solution to this problem is achieved according to the teaching of the independent claims. Various embodiments and further developments of the invention are the subject of the dependent claims.

[0009] A first aspect of the invention relates to a method, particularly a computer-implemented one, for recognizing a hand gesture. In this method, image data is acquired, the image data representing a two-dimensional image of a user's hand. Feature points are determined in the two-dimensional image of the hand, wherein the feature points comprise a wrist point and, for each finger of the hand, finger joint points and a fingertip point, wherein the finger joint points for each finger comprise an MCP joint point, a PIP joint point, and a DIP joint point. A hand position is then determined based on the relative positions of the feature points to one another in the two-dimensional image of the hand.

[0010] The aforementioned method, based on the first aspect, relies primarily on the fact that a hand position or gesture can be determined (solely) by the relative positions of feature points of the hand within the two-dimensional representation of the hand. This method directly utilizes the feature points in the two-dimensional representation of the hand to determine a hand position, thus eliminating the need for complex models, for example, for transformations into three-dimensional space. Since no 3D information is required, and hand position recognition relies solely on the 2D points, hand positions can be determined more quickly and accurately.

[0011] For hand posture recognition, distinctive 2D positions of the hand, the so-called keypoints, are determined. As will be explained in more detail below, this allows for the recognition of both bent and extended fingers, which in turn enables the recognition of a variety of hand gestures, such as an open hand, a pointing pose, or a pose for "two" (index and middle fingers extended, other fingers bent). It should be noted that numerous different methods for determining the keypoints of a hand are known, for example, using machine learning methods. Therefore, the recognition or determination of these points is assumed to be a known prerequisite for the present invention. These points are then further processed according to the method of the first aspect of the invention to recognize hand gestures.

[0012] The term "hand position" or "hand pose" used here refers specifically to the position of the fingers of one hand relative to each other, for example, whether individual fingers are bent or extended. Examples of hand positions include a flat hand, a clenched fist, a pointing pose in which only the index finger is extended, whether individual fingers touch, especially the thumb touching one of the other four fingers, and the like. A hand position can be called a "hand gesture," particularly if the hand gesture corresponds exactly to a hand position. The term "hand gesture" can also be understood as a (temporal) sequence of hand positions.

[0013] The term "keypoints" used here refers specifically to prominent points on a hand. Keypoints can be defined for each finger, particularly points at the fingertip and finger joints, especially at the distal interphalangeal joint, proximal interphalangeal joint, and metacarpophalangeal joint. The metacarpophalangeal joint (MCP joint) is the articulation between the metacarpal bones of the hand and the proximal phalanges. The proximal interphalangeal joint (PIP joint) and distal interphalangeal joint (DIP joint) are, accordingly, the joints between the phalanges. The phalanges can correspond (especially in a skeletal model of the hand) to bone vectors that connect adjacent joints of a finger.(the fingertip of a finger with the corresponding dip joint). Another characteristic point can be determined in the wrist area as the wrist point.

[0014] The term "detection device" used here refers in particular to a device that can detect objects in three-dimensional space without physical contact and record corresponding data, such as image data. In particular, the detection device can detect a user's hand. For example, optical methods can be used to detect a user's hand in space. Within the scope of the invention, the detection device can be, in particular, an image acquisition device. The "acquisition" of image data can refer specifically to recording the image data using the detection device, but alternatively also, for example, to acquiring, such as loading, existing image data.

[0015] The term "image capture device" used here refers specifically to a camera, particularly a digital camera. The camera can capture still images (photos) or moving images (videos), especially in the visible light (RGB) range or in the infrared (IR) range, particularly in the near-infrared (NIR) range. An IR image is well-suited for monitoring, for example, a vehicle interior, as it is robust against changing lighting conditions, such as strong sunlight. Recordings in darkness are also possible. It is understood that a suitable IR light source must be present. An image capture device can capture or record such images and output the corresponding image data. A sequence of several frames constitutes a moving image, particularly a video.

[0016] The term "vehicle" used here refers in particular to a passenger car, including all types of motor vehicles, hybrid and battery-powered electric vehicles, as well as vehicles such as sedans, vans, buses, trucks, delivery vans and the like.

[0017] Any terms used herein, such as "comprises," "includes," "features," "has," "with," or any other variant thereof, are intended to cover non-exclusive inclusion. For example, a method or apparatus that includes or features a list of elements is not necessarily limited to those elements but may include other elements not expressly listed or inherent in such method or apparatus.

[0018] Furthermore, unless explicitly stated otherwise, "or" refers to an inclusive or and not an exclusive "or". For example, a condition A or B is satisfied by one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

[0019] The terms "ein" or "eine," as used here, are defined as "one or more." The terms "ein anderer" and "ein Weitere," as well as any other variant thereof, are to be understood as "at least one more."

[0020] The term “plural” or “several”, as it may be used here, is to be understood in the sense of “two or more”.

[0021] The terms "configured" or "set up" to perform a specific function (and their respective variations) are understood within the meaning of the invention to mean that the corresponding device already exists in a configuration or setting in which it can perform the function, or at least that it is adjustable—i.e., configurable—so that it can perform the function after appropriate adjustment. Configuration can be achieved, for example, by adjusting parameters of a process sequence or by using switches or similar devices to activate or deactivate functionalities or settings. In particular, the device can have several predetermined configurations or operating modes, so that configuration can be carried out by selecting one of these configurations or operating modes.

[0022] Preferred embodiments of the method are described below, which, unless expressly excluded or technically impossible, can be combined with each other and with the other described aspects of the invention as desired.

[0023] In some embodiments, determining the hand position includes at least one of the following: An angle between two finger joints at at least one finger joint point can be determined. A distance between adjacent finger joint points and / or the fingertip point and the DIP joint point of a respective finger can be determined. A distance between the wrist point and at least one fingertip point and / or finger joint point can be determined. It is understood that, in particular, multiple angles and / or distances can be determined, up to and including all possible or relevant angles and / or distances that can be determined from the feature points. In particular, angles and / or distances can thus be determined as relative positional relationships of the feature points. From these relative positional relationships of the feature points, it can be deduced whether a finger (or several fingers) is bent or extended.are, from which a hand position can be inferred, as will be explained in more detail below.

[0024] In some embodiments, determining hand posture comprises determining an angle between two finger joints at at least one finger joint and comparing the determined angle with a threshold value, whereby a finger is determined to be extended if the angle is smaller than the at least one threshold value. In particular, several angles, especially up to three angles, can be determined for each finger, particularly at the MCP joint, the PIP joint, and / or the DIP joint. A threshold value can be, in particular, smaller than 180 degrees, for example, in the range of 150 to 160 degrees. This angle criterion is particularly meaningful in a lateral view of the hand, because if the angle at a finger joint is smaller than the threshold value, it is easy to see that the finger is not extended.The same threshold value can be assumed for the angles at the PIP joint and the DIP joint, since the individual finger joints cannot usually be moved independently for anatomical reasons anyway. If a finger cannot be clearly identified as not extended, the following additional criteria can be checked.

[0025] In some embodiments, determining the hand position includes determining a wrist distance between the wrist point and the MCP joint point of at least one finger. This wrist distance can, in a sense, serve as a reference for standardizing other specific distances, so that the determinations described below can be carried out independently of the hand's orientation in space and / or its distance from the camera.

[0026] In some embodiments, determining the hand position involves determining the distance between adjacent finger joint points and / or the fingertip point and the DIP joint point of a given finger, relating this distance to the determined wrist distance of the corresponding finger to obtain a corresponding distance relationship. It should be noted that this "distance criterion" can also be used to determine whether a finger is extended or not. When viewing the hand from the front, the previously determined angles may be approximately 180 degrees, even though a finger is not extended. The distances between the feature points may then appear "shortened." As explained above, relating these distances to the wrist distance allows them to be considered "relative" or "normalized."It should be noted that this provision can only be made for a finger if it could not previously be ruled out, based on the angles, that a finger is extended.

[0027] In some embodiments, the obtained distance relation is further compared with a corresponding distance relation derived from a known standard distance between adjacent finger joint points and / or the fingertip point and the DIP joint point of a respective finger. This standard distance is then related to a known standard wrist distance of the corresponding finger. A finger is determined to be not extended if the comparison of the distance relations reveals a difference (i.e., if the distance relations from the 2D image of the hand and the standard comparison data differ). Based on the absolute distances alone, it may not be possible to determine whether a finger is extended or not, i.e., whether a distance is actually shorter due to finger flexion or simply due to perspective. Therefore, a comparison with standard distances, i.e.,Measurements are taken using known, average bone lengths (or finger joint lengths), which always refer to an extended hand. If a difference is detected, a finger is therefore very likely not extended. It is understood that, due to measurement tolerances and for technical reasons, it may be stipulated that a difference is only recognized as such if it exceeds a predetermined threshold.

[0028] In some embodiments, determining the hand position includes at least determining whether at least one of the fingers of the hand is extended, whereby the finger is considered extended if the angle is greater than at least one threshold value and comparing the distances reveals no difference (or if the difference is less than a predetermined threshold value). In other words, if both an "angle criterion" and a "distance criterion" are met, a finger can be considered extended. It is understood that, in order to determine the hand position, it can be specifically determined which finger(s) of the hand is / are extended.

[0029] In some embodiments, determining the hand position includes determining whether the thumb is flexed or extended, whereby the relative position of at least one of the thumb's fingertip points, the DIP joint point, and the PIP joint point is determined relative to a line connecting the wrist point and the MCP joint point. The thumb can be considered separately due to its mobility compared to the other fingers. In particular, when flexed, the thumb can be placed over the palm. This can be determined by considering the thumb's finger joint points relative to a line connecting it to the index finger. A scalar product can be calculated for this purpose, which will be positive or negative depending on the thumb's position.

[0030] In some embodiments, determining hand position involves determining the distance between the wrist point and a fingertip point of a finger, and furthermore, determining whether the finger is flexed. The finger is considered flexed if the distance between the fingertip point and the wrist point is less than the distance between the corresponding MCP joint point and the wrist point. In other words, by determining and comparing these distances, it can be ascertained whether the fingertip is "on the palm," i.e., between the MCP joint and the wrist. From this, it can be concluded with a high degree of probability that the finger is flexed.

[0031] In some embodiments, determining the hand position further includes determining a distance between the wrist point and a DIP joint point of a finger, whereby the finger is only determined to be bent if the distance of the DIP joint point from the wrist point is also less than the distance of the corresponding PIP joint point from the wrist point. In other words, a finger is only recognized as (actually) bent if the DIP joint is closer to the wrist than the PIP joint.

[0032] While the previously mentioned criterion regarding the position of the fingertip may already be sufficient to determine whether a finger is bent, the second criterion just mentioned can increase the reliability of determining whether a finger is actually bent. For example, it can prevent (or at least reduce the risk of) unintentionally slightly bent (or not fully extended) fingers being interpreted as bent, thus misinterpreting a hand gesture and inadvertently and unintentionally controlling a vehicle function. Such a hand position can occur, for example, when gesturing, without the user intending a specific hand gesture.

[0033] It is understood that determining the hand position depends in particular on which finger(s) of the hand is / are bent. In particular, together with the information on which finger(s) of the hand is / are extended, a corresponding hand position can be determined.

[0034] In some embodiments, a distance is also determined between the tip of the thumb and the tip of one of the other fingers, particularly the index finger. For example, this can determine whether the thumb tip and index finger tip are approaching or touching, which can be useful for recognizing other hand gestures.

[0035] In some embodiments, a first hand position is determined at at least one initial time point and a second hand position at at least one subsequent time point, with a hand gesture being derived from the sequence of the first and second hand positions. While a single hand position can already represent a hand gesture, further hand gestures can be defined by a sequence of several hand positions. In other words, a change in a hand position can be determined as a hand gesture by determining hand positions at a plurality of successive time points. A sufficiently high temporal resolution, which is determined by a sampling rate of the detection device, is advantageous for capturing smooth movement. This can be, for example, 30 fps (frames per second).

[0036] For example, a single hand gesture could be a single pointing gesture. However, a hand gesture could also be defined by, for example, opening and closing a fist. By defining "static" hand gestures (individual hand positions) and "dynamic" hand gestures (sequences of hand gestures), diverse scenarios can be created, for example, to control a user interface in a vehicle with a range of vehicle functions.

[0037] A second aspect of the invention relates to a system for recognizing a hand gesture, comprising at least one processor configured to perform the method according to the first aspect. The system can, in particular, be configured to be installed in a vehicle. In this way, as described above, vehicle functions can be controlled by hand gestures.

[0038] In some embodiments, the system further comprises a detection device configured to capture a two-dimensional image of a user's hand. The detection device can, in particular, be an image capture device such as a camera, especially an infrared camera. As explained above, an infrared camera is particularly suitable because it can capture image data essentially independently of external lighting conditions, i.e., even in the dark.

[0039] A third aspect of the invention relates to a computer program with instructions which, when executed on a system according to the second aspect, cause the system to execute the method according to the first aspect.

[0040] The computer program can be stored, in particular, on a non-volatile data carrier. Preferably, this is a data carrier in the form of an optical data carrier or a flash memory module. This can be advantageous if the computer program itself is to be handled independently of a processor platform on which the one or more programs are to be executed. In another implementation, the computer program can exist as a file on a data processing unit, in particular on a server, and be downloadable via a data connection, for example, the Internet or a dedicated data connection, such as a proprietary or local network. Furthermore, the computer program can comprise a plurality of interacting individual program modules.

[0041] The system described in the second aspect may accordingly have a program memory in which the computer program is stored. Alternatively, the system may also be configured to access an external computer program, for example on one or more servers or other data processing units, via a communication link, in particular to exchange data with it that is used during the execution of the procedure or computer program or represents outputs of the computer program.

[0042] The features and advantages explained in relation to the first aspect of the invention also apply accordingly to the other aspects of the invention.

[0043] Further advantages, features and possible applications of the present invention will become apparent from the following detailed description in conjunction with the drawings.

[0044] This shows: Fig. 1 schematically a flat hand with characteristic points; Fig. 2. A hand in a pointing pose with characteristic points, using the index finger as an example; Fig. 3. A decision tree to determine whether a finger is extended; and Fig. 4 to Fig. 7 each a two-dimensional image of a hand in different hand positions, possibly from different perspectives.

[0045] The same reference numerals are used throughout the figures for the same or corresponding elements of the invention.

[0046] Fig. Figure 1 shows a hand 1 with corresponding feature points 2, 3, 4, 5, 6 ("keypoints"). There are numerous different methods for determining these points, particularly using machine learning, which are assumed to be known. The feature points 2, 3, 4, 5, 6 include, in particular, a wrist point 2 and, for each finger, corresponding finger joint points 3, 4, 5 and a fingertip point 6. The finger joint points 3, 4, 5 include, in particular, an MCP joint point 3 at the metacarpophalangeal joint ("MCP"), a PIP joint point 4 at the proximal interphalangeal joint ("PIP"), and a DIP joint point 5 at the distal interphalangeal joint ("DIP"). It is understood that these points can be determined equally for all fingers, i.e., thumb, index finger, middle finger, ring finger, and little finger.

[0047] Fig. Figure 2 shows a user's hand 1 in a pointing pose with feature points 2, 3, 4, 5, 6 marked, using the index finger as an example. The index finger is extended, while all other fingers are bent. As will be explained below, a hand position or hand gesture can be recognized solely based on the two-dimensional relative positions of feature points 2, 3, 4, 5, 6 of hand 1, which includes, in particular, angles and distances. This can be used as user input for gesture control in a vehicle or in other application scenarios.

[0048] The following section describes two different methods for determining whether a finger is extended and whether it is bent. These methods can then be applied to the different fingers to identify specific gesture characteristics. For example, an open hand can be recognized by five extended fingers. If all fingers are identified as extended, an open hand can be assumed, as shown in the following example. Fig. 1. Another example is the depiction of a pointing gesture with an extended index finger and three bent fingers (middle, ring, and little finger), as in Fig. 2 shown.

[0049] Essentially, the methodology for recognizing whether a finger is extended follows a decision tree 10, in which certain conditions 12, 13, 14 are checked, as in Fig. Figure 3 is shown. The individual conditions 12, 13, 14 are described in more detail below. The feature points 2, 3, 4, 5, 6 are assumed as input data 11 or determined from a two-dimensional image of the hand 1. Fig. Figure 4 shows a two-dimensional image of a flat hand with outstretched fingers, where the feature points and corresponding connecting lines (i.e., distances between the feature points) are marked. The image could, in particular, be an infrared image of the hand.

[0050] A first (necessary) condition 12 for an extended finger is that the angles between adjacent bones or finger joints are greater than a certain threshold. Therefore, it is checked whether the angles α, β, y at the finger joint points 3, 4, 5 are greater than certain thresholds, i.e., whether α > τ1, β > τ2, γ > τ2, with τ1 ∈ [0, 180], τ2 ∈ [0, 180]. If any of the three angles is smaller, extension can already be ruled out (result 16). Both in Fig. 5 as well as in Fig. In image 6, the index finger is bent, but the perspective is different. If you look at the user's hand 1 from the side, as in... Fig. As shown in Figure 5, this criterion is sufficient to determine that a finger is extended if all three angles are large enough. It becomes more difficult when the hand is positioned directly in front of the camera. Due to the 2D projection, the angles can be 180 degrees, even if the fingers are curved (see Figure 5). Fig. 6) Therefore, further conditions are examined to prove the presence of an extended finger (result 15).

[0051] If the hand is oriented frontally towards the camera, a relative distance criterion 13 is used to decide whether a finger is bent or extended. Fig. Figure 6 shows that the distances between the finger joint points 3, 4, and 5 (including the fingertip point 6) of the index finger are shortened because it is bent, while the angles are all 180 degrees. To illustrate this, it is first assumed that average human bone lengths are known for comparison purposes. In two dimensions, the Euclidean distance between two feature points in the case of an extended finger depends on the actual distance between the points and on both the distance of the hand to the camera and the orientation of the hand. To be as independent as possible from these last two conditions, the Euclidean distances of the individual finger segments are related to the distance between wrist point 2 and MCP joint point 3. This ratio is then compared to the same ratio of the known average bone lengths.If the two ratios ("distance relations") differ from each other, the presence of an extended finger can again be ruled out (Result 16). If the two ratios are close to each other, i.e., their difference is less than a threshold value, a final special case must be examined.

[0052] With a further condition 14, questions are to be examined which specifically relate to the thumb in camera directions where the angles separately satisfy the angle condition, even if the thumb is not necessarily extended.

[0053] To avoid this problem, a line is first defined at the MCP joint, given by the normal vector from MCP joint point 3 to wrist point 2. In the case of an extended thumb, PIP joint point 4, DIP joint point 5, and fingertip point 6 must lie on the negative side of the line at the MCP joint; that is, the dot product of the vector from the MCP joint to the corresponding feature point 4, 5, 6 with the normal vector must be less than zero. If any of the points 4, 5, or 6 lies on the positive side of the line, the thumb is not extended, as shown in [reference]. Fig. 7 shown.

[0054] Another important characteristic of gestures is the property of a finger to be fully flexed, as is the case, for example, in a fist. In most cases, it suffices to check whether the fingertip lies between the MCP joint and the wrist. This can be checked, for example, using Euclidean distances; that is, the distance from fingertip point 6 to wrist point 2 must be less than the distance from MCP joint point 3 to wrist point 2. Similarly, it can be further checked whether the DIP joint lies between the PIP joint and the wrist. If both conditions are met, the finger is fully flexed.

[0055] The two methods for determining extended or bent fingers can be used to identify a wide variety of hand positions, which can be part of numerous gestures. Furthermore, additional characteristics can be determined to recognize other hand gestures. For example, an "okay" pose (thumb touching index finger, other fingers extended) can also be identified by applying the extended finger test to the middle, ring, and little fingers and adding a test of the Euclidean distance between the tip of the thumb and the tip of the index finger.

[0056] The hand gesture recognition method can be used, in particular, in a vehicle to enable contactless interaction with a user interface. In this case, an IR sensor (an IR camera) can be used to capture data from the driver or another passenger, along with any machine learning method to determine the feature points (especially their 2D positions) of the hand. From this, various hand gestures can then be recognized, as explained.

[0057] For example, if the thumb is detected as extended and the other fingers as bent, a thumb gesture can be recognized. Since the camera's position in the vehicle is known, the direction of the thumb can also be determined, distinguishing between left and right. This gesture can then be used, for example, to change radio stations forward or backward.

[0058] A hand position can also be a pointing pose, in which the index finger of the detected hand is extended. It should be noted that "extended" does not necessarily mean a fully extended finger, but rather that the extension exceeds a predefined value; the finger may still be slightly bent, but is essentially fully extended. A pointing pose is recognized, in particular, when at least the middle, ring, and little fingers of the hand are bent simultaneously. The thumb can be in any position. A pointing pose can indicate that the user wishes to enter information on the user interface.

[0059] Furthermore, the described method can also be used to recognize specific sequences of hand gestures. For example, a double-five gesture consists of opening the hand twice. Since a fist can be recognized as a hand gesture (pose) (all fingers bent), as can an open hand (all fingers extended), a sequence of images can now be analyzed to identify the desired hand gesture for interaction with the user interface.

[0060] While at least one exemplary embodiment has been described above, it should be noted that a large number of variations exist. It should also be noted that the described exemplary embodiments are merely non-limiting examples, and it is not intended to restrict the scope, applicability, or configuration of the devices and methods described herein. Rather, the preceding description will provide the person skilled in the art with guidance for implementing at least one exemplary embodiment. It is understood that various modifications to the function and arrangement of the elements described in an exemplary embodiment can be made without derogating from the subject matter defined in the appended claims and their legal equivalents. REFERENCE MARK LIST 1 hand 2 Wrist point 3rd Finger Joint Point (MCP) 4 Finger joint point (PIP) 5 Finger joint point (DIP) 6 Fingertip point 10 Methodology for determining whether a finger is extended 11-16 steps / processes / data / results within the framework of methodology 10

Claims

[1] Method for recognizing a hand gesture, the method comprising: - Capturing image data, wherein the image data represents a two-dimensional image of a user's hand (1); - Determining feature points (2, 3, 4, 5, 6) in the two-dimensional image of the hand (1), wherein the feature points (2, 3, 4, 5, 6) include a wrist point (2) and for the fingers of the hand (1) each finger joint points (3, 4, 5) and a fingertip point (6), wherein the finger joint points (3, 4, 5) for each finger each include an MCP joint point (3), a PIP joint point (4) and a DIP joint point (5); - Determining a hand position based on the relative positional relationships of the feature points (2, 3, 4, 5, 6) to each other in the two-dimensional image of the hand 12). [2] The method of claim 1, wherein determining the hand position comprises at least one of the following: - Determining an angle between two finger joints at at least one finger joint point (3, 4, 5), - Determining a distance between adjacent finger joint points (3, 4, 5) and / or the fingertip point (6) and the DIP joint point (5) of a respective finger, and - Determining a distance between the wrist point (2) and at least one fingertip point (6) and / or finger joint point (3, 4, 5). [3] Method according to claim 1 or 2, wherein determining the hand position comprises determining an angle between two finger segments at at least one finger joint point (3, 4, 5) and comparing the determined angle with a threshold value, wherein a finger is determined to be not extended if the angle is smaller than the at least one threshold value. [4] Method according to any of the preceding claims, wherein determining the hand position comprises determining a wrist distance between the wrist point (2) and the MCP joint point (3) of at least one finger. [5] Method according to claim 4, wherein determining the hand position comprises determining a distance between adjacent finger joint points (3, 4, 5) and / or the fingertip point (6) and the DIP joint point (5) of a respective finger, wherein the distance is related to the determined wrist distance of the corresponding finger in order to obtain a corresponding distance relation. [6] Method according to claim 5, further comprising comparing the obtained distance relation with a corresponding distance relation obtained from a known standard distance between adjacent finger joint points (3, 4, 5) and / or the fingertip point (6) and the DIP joint point (5) of a respective finger, which is related to a known standard wrist distance of the corresponding finger, wherein a finger is determined to be not extended if the comparison of the distance relations reveals a difference. [7] Method according to claims 3 and 6, wherein determining the hand position includes at least determining whether at least one of the fingers of the hand (1) is extended, wherein the finger is determined to be extended if the angle is greater than the at least one threshold value and comparing the distance relations reveals no difference. [8] Method according to one of the preceding claims, wherein determining the hand position comprises determining whether the thumb is bent or extended, wherein a relative position of at least one of the fingertip point (6), the DIP joint point (5) and the PIP joint point (4) of the thumb is determined relative to a connecting line between the wrist point (2) and the MCP joint point (3). [9] Method according to any of the preceding claims, wherein determining the hand position comprises determining a distance between the wrist point (2) and a fingertip point (6) of a finger and further comprising determining whether the finger of the hand (1) is bent, wherein the finger is determined to be bent if a distance of the fingertip point (6) from the wrist point (2) is less than a distance of the corresponding MCP joint point (3) from the wrist point (2). [10] Method according to claim 9, wherein determining the hand position further comprises determining a distance between the wrist point (2) and a DIP joint point (5) of a finger, wherein the finger is determined to be bent only if a distance of the DIP joint point (5) from the wrist point (2) is also less than a distance of the corresponding PIP joint point (4) from the wrist point (2). [11] Method according to one of the preceding claims, further comprising determining a distance between the fingertip point (6) of the thumb and the fingertip point (6) of one of the other fingers, in particular the index finger. [12] Method according to any one of the preceding claims, further comprising: - Determining an initial hand position at at least one initial point in time; - Determining a second hand position at at least one second time point; and - Determining a hand gesture from the sequence of the first and second hand positions. [13] System for recognizing a hand gesture, comprising at least one processor configured to perform the method according to any of the preceding claims. [14] System according to claim 12, further comprising a capture device which is configured to capture a two-dimensional image of a user's hand (1). [15] Computer program with instructions which, when executed on a system according to claim 13 or 14, cause the system to execute the method according to any one of claims 1 to 12.

Images