Gesture recognition method and device, computer equipment and storage medium

Abstract

The invention relates to a gesture recognition method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring gesture space information of a current gesture based on a received gesture recognition instruction; the gesture space information comprises three-dimensional coordinates of fingers and a wrist; obtaining a first serialization feature of the current gesture based on the three-dimensional coordinates of the fingers and the wrist; sequentially calculating gesture matching degrees of the first serialized features and second serialized features of a plurality of preset gestures; and determining a gesture recognition result of the current gesture based on the gesture matching degree. By adopting the method, the matching of the current gesture and the preset gesture can be realized according to the processing of the characteristics of fingers, wrists and the like, the gesture recognition result can be obtained without a neural network or machine learning training recognition, and the effect of improving the gesture recognition speed and accuracy is achieved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a gesture recognition method, apparatus, computer equipment and storage medium. Background Art

[0002] With the development of science and technology, the types of terminal devices are constantly expanding, and the functions of terminals are becoming more and more diversified. Virtual Reality and Augmented Reality are part of the diversified functions. Virtual Reality uses computers to simulate the environment so that users can immerse themselves in the environment. Augmented Reality simulates virtual information and applies it to images in the real world to enhance the real world. Virtual Reality and Augmented Reality rely on the interaction between the virtual and the real, and require the recognition of sounds, images, texts, actions, etc.

[0003] Traditional techniques for gesture recognition rely mainly on neural networks and machine learning. It is necessary to establish a relatively complete set of gesture samples and split them into training sets and test sets for machine learning. In order to achieve accurate recognition, each gesture interaction behavior requires a large number of sample sets for training, resulting in long training time and large amounts of computational data.

[0004] Based on this, traditional gesture recognition technology still has the problems of long recognition time and large amount of calculation data. Summary of the invention

[0005] Based on this, it is necessary to provide a gesture recognition method, apparatus, computer device and storage medium that can reduce recognition time and calculation data volume in response to the above technical problems.

[0006] In a first aspect, this embodiment provides a gesture recognition method, the method comprising:

[0007] Acquire gesture space information of the current gesture based on the received gesture recognition instruction; the gesture space information includes three-dimensional coordinates of the fingers and wrist;

[0008] Based on the three-dimensional coordinates of the finger and the wrist, obtain a first serialized feature of the current gesture;

[0009] sequentially calculating gesture matching degrees of the first serialized feature and second serialized features of a plurality of preset gestures;

[0010] A gesture recognition result of the current gesture is determined based on the gesture matching degree.

[0011] In one embodiment, acquiring gesture space information of the current gesture based on the received gesture recognition instruction includes:

[0012] Based on the received gesture recognition instruction, acquiring a depth image captured by an image acquisition device;

[0013] Recognize hand features of the current gesture based on the depth image, the hand features including pixel coordinates of fingers and wrists;

[0014] Based on the pixel coordinates, the three-dimensional coordinates of the fingers and wrist are determined.

[0015] In one embodiment, the gesture includes a static gesture, and obtaining the first serialized feature of the current gesture based on the three-dimensional coordinates of the finger and the wrist includes:

[0016] Based on the three-dimensional coordinates of the finger and the wrist, determining a plurality of feature point pairs between the finger and the wrist;

[0017] Calculating feature point distances of a plurality of feature point pairs;

[0018] Based on the multiple feature point distances, a first serialized feature of the static gesture is determined.

[0019] In one embodiment, the second serialization feature includes a second serialization feature of a preset static gesture, and the sequentially calculating the gesture matching degree between the first serialization feature and the second serialization features of a plurality of preset gestures includes:

[0020] Determining feature point distance differences of multiple groups of feature point pairs based on the first serialized feature and the second serialized feature of the preset static gesture;

[0021] Based on the feature point distance differences, calculating feature score values ​​of multiple groups of feature point pairs;

[0022] The gesture matching degree is obtained according to the feature score values ​​of the plurality of feature point pairs.

[0023] In one embodiment, the gesture includes a dynamic gesture, and obtaining the first serialized feature of the current gesture based on the three-dimensional coordinates of the finger and the wrist includes:

[0024] Taking any finger or wrist as a feature point, obtaining the three-dimensional coordinates of the feature point at multiple times;

[0025] Based on the three-dimensional coordinates of the feature point at multiple moments, calculating the feature point displacement of the feature point within multiple adjacent moments;

[0026] Based on the feature point displacement, a first serialized feature of the dynamic gesture is determined.

[0027] In one of the embodiments, determining the first serialized feature of the dynamic gesture based on the feature point displacement further includes:

[0028] Calculate the normalized sampling time and the normalized sampling interval of the dynamic gesture within a plurality of adjacent moments based on the action start time and the action end time of the dynamic gesture and the initial sampling interval of the adjacent moments;

[0029] Calculating the normalized displacements of the feature points in the plurality of adjacent moments based on the normalized sampling moment, the normalized sampling interval, the displacements of the feature points in the adjacent moments, and the total displacements of the feature points of the dynamic gesture;

[0030] Based on the normalized displacement of the feature points in the plurality of adjacent moments, a first serialized feature of the dynamic gesture is obtained.

[0031] In one embodiment, the second serialization feature includes a second serialization feature of a preset dynamic gesture, and the sequentially calculating the gesture matching degree between the first serialization feature and the second serialization features of a plurality of preset gestures includes:

[0032] Determine a feature point displacement difference of a plurality of feature point displacements based on the first serialized feature and the second serialized feature of the preset dynamic gesture;

[0033] Calculating feature score values ​​of the displacements of a plurality of feature points based on the feature point displacement differences;

[0034] The gesture matching degree is obtained according to the feature score values ​​of the displacements of the plurality of feature points.

[0035] In a second aspect, this embodiment provides a gesture recognition device, the device comprising:

[0036] An acquisition module, configured to acquire gesture space information of a current gesture based on a received gesture recognition instruction; the gesture space information includes three-dimensional coordinates of fingers and wrists;

[0037] A feature module, used for obtaining a first serialized feature of the current gesture based on the three-dimensional coordinates of the finger and the wrist;

[0038] A matching module, used for sequentially calculating the gesture matching degree of the first serialized feature and the second serialized features of a plurality of preset gestures;

[0039] A determination module is used to determine a gesture recognition result of the current gesture based on the gesture matching degree.

[0040] In a third aspect, this embodiment provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0041] In a fourth aspect, this embodiment provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0042] The above-mentioned gesture recognition method, device, computer equipment and storage medium obtain gesture space information of the current gesture based on the received gesture recognition instruction; the gesture space information includes the three-dimensional coordinates of the fingers and wrist; based on the three-dimensional coordinates of the fingers and wrist, the first serialized feature of the current gesture is obtained; the gesture matching degree of the first serialized feature and the second serialized features of multiple preset gestures is calculated in turn; based on the gesture matching degree, the gesture recognition result of the current gesture is determined. As a gesture recognition method that does not require neural network or machine learning training and recognition, it can achieve matching of the current gesture with the preset gesture based on the processing of features such as fingers and wrists to obtain a gesture recognition result, which can reduce recognition time and calculation data volume, and achieve the effect of improving gesture recognition speed and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A diagram of an application environment of a gesture recognition method in an embodiment;

[0044] Figure 2 is a flowchart of a gesture recognition method in one embodiment;

[0045] Figure 3 is a flowchart of a gesture recognition method in another embodiment;

[0046] Figure 4 is a flowchart of a gesture recognition method in another embodiment;

[0047] Figure 5 is a flowchart of a gesture recognition method in another embodiment;

[0048] Figure 6 is a schematic diagram of a first serialization feature in one embodiment;

[0049] Figure 7 is a schematic diagram of a first serialization feature in another embodiment;

[0050] Figure 8 is a structural block diagram of a gesture recognition device in one embodiment;

[0051] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] The gesture recognition method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can obtain the gesture space information of the current gesture based on the received gesture recognition instruction; based on the three-dimensional coordinates of the finger and the wrist, obtain the first serialized feature of the current gesture; obtain the second serialized feature of the preset gesture by communicating with the server 104, and calculate the gesture matching degree of the first serialized feature and the second serialized features of multiple preset gestures in turn; determine the gesture recognition result of the current gesture based on the gesture matching degree. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0054] In one embodiment, Figure 2 As shown, a gesture recognition method is provided, which is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0055] Step S100: acquiring gesture space information of the current gesture based on the received gesture recognition instruction.

[0056] Among them, the gesture recognition instruction can be issued by the user, for example, the gesture recognition instruction can be determined by identifying the user's voice, picture, text, action behavior, or it can be issued by the user based on an input device. It can also be determined when the scene meets the preset triggering conditions. It can also be other gesture recognition instruction generation methods based on the data collected by the terminal 102. This embodiment does not limit this.

[0057] The current gesture is the gesture to be recognized, and the gesture spatial information includes the three-dimensional coordinates of the fingers and wrists. The gesture spatial information can be acquired by an image acquisition device. Based on the gesture spatial information acquired by the image acquisition device, the position of the hand feature points in space can be quickly identified by an existing tool set, which will not be elaborated in this article.

[0058] The fingers may include a thumb, an index finger, a middle finger, a ring finger and a little finger. Further, the three-dimensional coordinates of the fingers may also include the three-dimensional coordinates of each joint point of each finger.

[0059] Step S200, obtaining a first serialized feature of the current gesture based on the three-dimensional coordinates of the fingers and the wrist.

[0060] Among them, the first serialized feature is obtained based on the three-dimensional coordinates of at least one of the fingers and the wrist. It should be understood that in some cases, the terminal 102 cannot completely collect the three-dimensional coordinates of the fingers and wrists through the sensor. For example, when the current gesture is a fist, and the back of the hand is facing the sensor side, the gesture spatial information collected by the sensor may only obtain the three-dimensional coordinates of the wrist after analysis. Therefore, when the joints such as fingers cannot be identified, auxiliary gesture recognition can be performed based on the collected images of the wrist and the current gesture.

[0061] The first serialized feature may be a three-dimensional coordinate of at least one of a finger and a wrist, or may be obtained by converting the three-dimensional coordinate of at least one of a finger and a wrist according to a preset rule.

[0062] Step S300: sequentially calculating gesture matching degrees between the first serialized feature and second serialized features of a plurality of preset gestures.

[0063] Among them, the preset gesture can be pre-set by the user. Accordingly, the second serialization feature of the pre-set gesture can be performed by referring to the above steps. For example, the gesture spatial information of the gesture to be entered can be obtained, and based on the three-dimensional coordinates of the fingers and wrists, the second serialization feature of the preset gesture can be obtained.

[0064] The gesture matching degree of the first serialized feature and the second serialized feature of the plurality of preset gestures is calculated in sequence, and each feature value in the first serialized feature is matched with the feature value in the second serialized feature one by one to obtain the comprehensive matching result of the current gesture and the single preset gesture, thereby calculating the matching degree of the plurality of preset gestures to obtain the gesture matching degree of the current gesture and the plurality of preset gestures. It can be understood that the gesture matching degree can be calculated based on the degree of deviation, and accordingly, the lower the deviation value, the higher the matching degree. The gesture matching degree can also be calculated based on the similarity, and accordingly, the higher the similarity, the higher the matching degree.

[0065] Step S400: determining a gesture recognition result of the current gesture based on the gesture matching degree.

[0066] Determining the gesture recognition result based on the gesture matching degree may be based on the gesture matching degree between the current gesture and a plurality of preset gestures, and taking the preset gesture with the highest matching degree as the gesture recognition result of the current gesture.

[0067] Furthermore, after determining the gesture recognition result of the current gesture, the related operations included in the preset gesture may be executed according to the preset gesture determined by the recognition.

[0068] A gesture recognition method provided in this embodiment obtains gesture space information of a current gesture based on a received gesture recognition instruction; the gesture space information includes three-dimensional coordinates of fingers and wrists; based on the three-dimensional coordinates of the fingers and wrists, a first serialized feature of the current gesture is obtained; gesture matching degrees of the first serialized feature and second serialized features of multiple preset gestures are calculated in turn; and a gesture recognition result of the current gesture is determined based on the gesture matching degrees. As a gesture recognition method that does not require neural network or machine learning training and recognition, the current gesture can be matched with a preset gesture based on the processing of features such as fingers and wrists to obtain a gesture recognition result, which can reduce recognition time and the amount of data calculation, thereby achieving the effect of improving gesture recognition speed and accuracy.

[0069] In one embodiment, acquiring gesture space information of the current gesture based on the received gesture recognition instruction includes:

[0070] Based on the received gesture recognition instruction, acquiring a depth image captured by an image acquisition device;

[0071] Recognize hand features of the current gesture based on the depth image, the hand features including pixel coordinates of fingers and wrists;

[0072] Based on the pixel coordinates, the three-dimensional coordinates of the fingers and wrist are determined.

[0073] The image acquisition device may be a depth sensor or a multi-camera system for locating spatial positions. Furthermore, the depth sensor may be one or more of an infrared depth sensor, a stereoscopic vision sensor, an ultrasonic sensor, a time-of-flight sensor, a lidar sensor, etc. The image acquisition device may be arranged on a terminal. Taking a head-mounted display device as an example, the image acquisition device may be arranged around the head-mounted display device to collect surrounding scenes including user gestures.

[0074] Recognizing the hand features of the current gesture based on the depth image can be performed based on an existing image recognition algorithm or tool set. Before recognizing the hand features of the current gesture based on the depth image, it can also be determined whether there are hand features in the depth image, and if so, a hand feature recognition operation is performed.

[0075] Based on the pixel coordinates, the three-dimensional coordinates of the fingers and wrist are determined. The three-dimensional coordinates of the fingers and wrist can be determined based on the pixel coordinates of one or more images. Based on the pixel coordinates of a single image, the three-dimensional coordinates of the fingers and wrist can be obtained based on the pixel coordinates and other data, such as depth data such as laser ranging and infrared ranging. It can also be that the three-dimensional structure of the hand is predicted by a pre-trained artificial intelligence model. In a specific embodiment, the three-dimensional coordinates can be determined by using a multi-level pixel alignment implicit function PIFuHD. The three-dimensional coordinates of the fingers and wrist are determined based on the pixel coordinates of multiple images. It can be determined in combination with depth data, or by tracking hand images at different angles in multiple images to determine the three-dimensional coordinates of the fingers and wrist.

[0076] Furthermore, after determining the three-dimensional coordinates of the fingers and wrist, the method further includes: taking the three-dimensional coordinates of the wrist as the origin, resetting the three-dimensional coordinates of the fingers, and obtaining calibrated three-dimensional coordinates of the fingers and wrist.

[0077] A gesture recognition method provided in this embodiment can realize the recognition of hand features by recognizing the pixel coordinates of fingers and wrists based on depth images, and then obtaining the three-dimensional coordinates of the fingers and wrists based on the pixel coordinates. After the hand features are recognized, the three-dimensional coordinates are calibrated with the three-dimensional coordinates of the wrist as the origin, and inaccurate gesture recognition caused by the overall position offset of the gesture can be avoided, thereby achieving the effect of improving the accuracy of gesture recognition.

[0078] In one embodiment, obtaining the first serialized feature of the current gesture based on the three-dimensional coordinates of the finger and the wrist includes:

[0079] Based on the three-dimensional coordinates of the finger and the wrist, determining a plurality of feature point pairs between the finger and the wrist;

[0080] Calculating feature point distances of a plurality of feature point pairs;

[0081] Based on the multiple feature point distances, a first serialized feature of the static gesture is determined.

[0082] Wherein, the gesture includes a static gesture. Based on the three-dimensional coordinates of the fingers and the wrist, a plurality of feature point pairs between the fingers and the wrist are determined, which can be a combination of the feature points of the five fingers and the wrist. Specifically, the feature points of the five fingers and the wrist can be combined in pairs to obtain 15 feature point pairs including wrist-thumb, wrist-index finger, wrist-middle finger, wrist-ring finger, wrist-little finger, thumb-index finger, thumb-middle finger, thumb-ring finger, thumb-little finger, index finger-middle finger, index finger-ring finger, index finger-little finger, middle finger-ring finger, middle finger-little finger and ring finger-little finger. In actual operation, the five fingers may not be fully recognized and 15 feature point pairs cannot be obtained. Therefore, the recognized fingers and wrists can be arranged in a preset order to obtain the combination result of the feature point pairs of the recognized feature points. For example, when one feature point is missing, 10 groups of feature point pairs can be obtained by combination. When two feature points are missing, 6 groups of feature point pairs can be obtained by combination, and so on.

[0083] The feature point distances of the plurality of feature point pairs are calculated, that is, the distances are calculated according to the three-dimensional coordinates of the feature points in the feature point pairs, the distances between the feature points are obtained, and the feature point distances corresponding to each feature point pair are obtained.

[0084] Based on the distances of multiple feature points, the first serialized feature of the static gesture is determined, and the feature point distances may be arranged in a preset order to obtain the arranged first serialized feature.

[0085] The present embodiment provides a gesture recognition method, which obtains a first serialized feature by calculating the feature point distances of multiple feature point pairs of the finger and the wrist, and can realize the calculation of the relative positions between the multiple feature points of the finger and the wrist. It can be understood that the relative distance can avoid the problem of low recognition rate caused by the overall offset of the three-dimensional coordinates. Based on the uniqueness between the relative distance feature and the gesture, the relative distance can better represent the gesture itself and be used for matching calculations, thereby achieving the effect of improving the accuracy of gesture recognition.

[0086] In one embodiment, the sequentially calculating the gesture matching degree of the first serialized feature and the second serialized features of the plurality of preset gestures comprises:

[0087] Determining feature point distance differences of multiple groups of feature point pairs based on the first serialized feature and the second serialized feature of the preset static gesture;

[0088] Based on the feature point distance differences, calculating feature score values ​​of multiple groups of feature point pairs;

[0089] The gesture matching degree is obtained according to the feature score values ​​of the plurality of feature point pairs.

[0090] Among them, the second serialization feature includes the second serialization feature of the preset static gesture. The second serialization feature of the preset static gesture can be preset by the user. Accordingly, the second serialization feature of the preset gesture can be preset by referring to the above steps. For example, the gesture space information of the gesture to be recorded can be obtained to obtain the three-dimensional coordinates of the finger and wrist, multiple feature point pairs are determined based on the three-dimensional coordinates of the finger and wrist, and the second serialization feature of the preset gesture is determined based on the feature point distance calculated based on the multiple feature point pairs.

[0091] Based on the first serialized feature and the second serialized feature of the preset static gesture, the feature point distance difference of multiple groups of feature point pairs is determined, which can be to calculate the difference between the distances of two feature points of the same feature point pair in the first serialized feature and the second serialized feature. For example, when the feature point pair is obtained based on the pairwise combination of feature points of fingers and wrists, the feature point pair distance difference of the same feature point pair can represent the distance difference between the current static gesture and the preset static gesture at a specific feature point pair.

[0092] Based on the feature point distance difference, the feature score values ​​of multiple groups of feature point pairs are calculated, and the corresponding feature score values ​​can be assigned according to the size of the distance difference. The assigned feature score value can be converted according to a certain coefficient based on the distance, or it can be assigned in a gradient manner according to different distance ranges.

[0093] The gesture matching degree is obtained according to the feature score values ​​of the multiple feature point pairs. The gesture matching degree can be obtained by summing up the feature score values ​​of the multiple feature point pairs. Furthermore, the percentage can be calculated based on the ratio of the sum value to the full score value, and the percentage is used as the value of the gesture matching degree.

[0094] A gesture recognition method provided in this embodiment calculates the difference between the distance of each feature point in the first serialized feature and the second serialized feature, so as to calculate the matching degree of each pair of corresponding feature points, thereby calculating the gesture matching degree of the entire gesture, thereby achieving the effect of improving the accuracy of gesture recognition.

[0095] In one embodiment, obtaining the first serialized feature of the current gesture based on the three-dimensional coordinates of the finger and the wrist includes:

[0096] Taking any finger or wrist as a feature point, obtaining the three-dimensional coordinates of the feature point at multiple times;

[0097] Based on the three-dimensional coordinates of the feature point at multiple moments, calculating the feature point displacement of the feature point within multiple adjacent moments;

[0098] Based on the feature point displacement, a first serialized feature of the dynamic gesture is determined.

[0099] The gesture includes a dynamic gesture, which may be a hand movement made by a user within the terminal acquisition range, such as waving, pushing, pulling, opening, closing, turning, etc. In this embodiment, the first serialized feature may be obtained based on the displacement of a single gesture feature point at different times.

[0100] Any finger or wrist is used as a feature point, and the selection of feature points can be arbitrary or based on a preset selection order. In a specific embodiment, the preset selection order can be the index finger, thumb, middle finger, ring finger, little finger and wrist are selected in sequence. When the feature point of the index finger is not recognized, the next recognizable feature point is used as the feature point required for the first serialized feature.

[0101] To obtain the three-dimensional coordinates of the feature point at multiple moments, multiple images of the current dynamic gesture may be collected based on a preset unit time, thereby obtaining the three-dimensional coordinates of the feature point at multiple moments based on feature point recognition.

[0102] Based on the three-dimensional coordinates of the feature points at multiple moments, the feature point displacements of the feature points in multiple adjacent moments are calculated. The adjacent moments can be separated by a preset unit time, or the motion trajectory of the current gesture can be fitted based on the three-dimensional coordinates of multiple moments, and the preset number of moments are resampled based on the motion time of the entire dynamic gesture to obtain the three-dimensional coordinates of the resampled multiple moments. At this time, the adjacent moments can be separated by the resampled unit time. The calculation of the feature point displacement can be to calculate the distance between the feature point coordinates of the previous moment and the next moment in the adjacent moments. The feature point displacement can be expressed in the form of a three-dimensional vector.

[0103] Based on the feature point displacement, the first serialization feature of the dynamic gesture is determined, and the feature point displacements obtained in multiple adjacent moments may be arranged according to a preset rule to obtain the first serialization feature of the dynamic gesture.

[0104] A gesture recognition method provided in this embodiment can realize motion capture of dynamic gestures by acquiring the three-dimensional coordinates of any feature point of a dynamic gesture at multiple moments and obtaining a first serialized feature based on the feature point displacement of the feature point at multiple adjacent moments. Feature points are selected by using preset feature point selection rules, which can also avoid gesture recognition errors caused by the inability to recognize some feature points, thereby achieving the effect of improving the accuracy of gesture recognition.

[0105] In one of the embodiments, determining the first serialized feature of the dynamic gesture based on the feature point displacement further includes:

[0106] Calculate the normalized sampling time and the normalized sampling interval of the dynamic gesture within a plurality of adjacent moments based on the action start time and the action end time of the dynamic gesture and the initial sampling interval of the adjacent moments;

[0107] Calculating the normalized displacements of the feature points in the plurality of adjacent moments based on the normalized sampling moment, the normalized sampling interval, the displacements of the feature points in the adjacent moments, and the total displacements of the feature points of the dynamic gesture;

[0108] Based on the normalized displacement of the feature points in the plurality of adjacent moments, a first serialized feature of the dynamic gesture is obtained.

[0109] The action start time can be the time when the dynamic gesture starts to move, or enters the collection range, or other action start time that is unified with the user through interaction; correspondingly, the action end time can be the time when the dynamic gesture ends the movement and becomes still, or leaves the collection range, or other action end time that is unified with the user through interaction. It can be understood that the unity with the user through interaction can be set by the user, or other settings based on needs, so that the user can make corresponding actions within a certain time range, and the start time and end time of the certain time range can be used as the action start time and action end time.

[0110] The sensor of the terminal may perform sampling at a preset unit time without being affected by the movement duration of the dynamic gesture. At this time, the initial sampling interval between adjacent moments may be the preset unit time.

[0111] It is understandable that different users may complete actions at different times and with different amplitudes, and therefore the time and amplitude of dynamic gesture movements need to be unified.

[0112] In this embodiment, the unification of time includes: calculating the normalized sampling time and the normalized sampling interval of the dynamic gesture in the plurality of adjacent moments. The normalized sampling time can be calculated based on the ratio of the cumulative motion duration of the dynamic gesture at each moment to the actual total motion duration, multiplied by the preset motion duration, to obtain the normalized sampling time of each moment. Accordingly, the normalized sampling interval can be calculated based on the normalized sampling time of each adjacent moment.

[0113] The unification of amplitude is the unification in space, including: based on the normalized sampling time, the normalized sampling interval, the displacement of the feature points in the adjacent time, and the total displacement of the feature points of the dynamic gesture, the normalized displacement of the feature points in the multiple adjacent time is calculated. The normalized displacement of the feature points can be calculated by multiplying the ratio of the displacement of the feature points of the dynamic gesture at each adjacent time to the total displacement of the feature points by the preset feature point displacement distance to obtain the normalized normalized displacement of each feature point.

[0114] Furthermore, after the collected dynamic gestures are normalized in time and space, the sampling numbers may be different due to the different completion times of the actions. Therefore, after calculating the normalized displacements of the feature points in the multiple adjacent moments, the method further includes: fitting the motion trajectory of the dynamic gesture based on the action start time, the action end time and the normalized displacements of the feature points in the multiple adjacent moments; resampling the normalized displacements of the feature points of the dynamic gesture based on the motion trajectory and the preset sampling number to obtain the normalized displacements of the feature points in the multiple adjacent moments after resampling. The multiple adjacent moments after resampling may be multiple moments determined based on the preset sampling number.

[0115] Based on the normalized displacements of the feature points in the adjacent moments, the first serialized feature of the dynamic gesture is obtained. The normalized displacements of the feature points are arranged in a preset order to obtain the first serialized feature of the dynamic gesture.

[0116] A gesture recognition method provided in this embodiment normalizes dynamic gestures in time and space based on the action start time, action end time, initial sampling intervals of adjacent moments, feature point displacements within adjacent moments, and total feature point displacements of the dynamic gestures, thereby achieving unification of the specification of serialized features between the current gesture and the preset gesture, avoiding the influence of the gesture recognition on the inconsistency of the dynamic gesture movement duration or amplitude, and achieving the effect of improving the gesture recognition accuracy.

[0117] In one embodiment, the sequentially calculating the gesture matching degree of the first serialized feature and the second serialized features of the plurality of preset gestures comprises:

[0118] Determine a feature point displacement difference of a plurality of feature point displacements based on the first serialized feature and the second serialized feature of the preset dynamic gesture;

[0119] Calculating feature score values ​​of the displacements of a plurality of feature points based on the feature point displacement differences;

[0120] The gesture matching degree is obtained according to the feature score values ​​of the displacements of the plurality of feature points.

[0121] Wherein, the second serialized feature includes a second serialized feature of a preset dynamic gesture. Determining the feature point displacement difference of the plurality of feature point displacements may be based on the difference calculated based on the feature point displacement of the corresponding positions of the first serialized feature and the second serialized feature. The difference in the feature point displacement of the corresponding positions may represent the displacement difference between the feature points of the current dynamic gesture and the preset dynamic gesture at the same time.

[0122] Based on the feature point displacement difference, the feature score values ​​of the feature point displacements are calculated, and the corresponding feature score value can be assigned according to the size of the displacement difference. The assigned feature score value can be converted according to a certain coefficient based on the displacement difference, or it can be assigned in a gradient manner according to different displacement ranges.

[0123] The gesture matching degree is obtained according to the feature score values ​​of the displacements of the multiple feature points. The gesture matching degree can be obtained by summing up the feature score values ​​based on the displacements of the multiple feature points. Furthermore, the percentage can be calculated based on the ratio of the sum value to the full score value, and the percentage is used as the value of the gesture matching degree.

[0124] A gesture recognition method provided in this embodiment calculates the difference between the displacement of each feature point in the first serialized feature and the second serialized feature, thereby calculating the matching degree of each pair of corresponding feature point displacements, thereby calculating the gesture matching degree of the entire dynamic gesture, which can achieve the effect of improving the accuracy of gesture recognition.

[0125] In order to more clearly illustrate the technical solution of the present application, the present application also provides a detailed embodiment.

[0126] In this embodiment, if Figure 3 As shown, the gesture recognition method of this embodiment includes:

[0127] (1) Gesture presets

[0128] In this embodiment, the gesture types include static gestures and dynamic gestures.

[0129] The user interacts with the terminal, sends a command to add a gesture, and selects a static gesture or a dynamic gesture. The terminal prompts you to enter a specific gesture to start the gesture recognition process. The specific gesture can be a gesture provided by the terminal or set by the user to start and end gesture recognition.

[0130] Determine whether a specific gesture is input. If not, continue to prompt for a specific gesture to start the process; if yes, record the finger joints, wrist position information, time information and gesture type information.

[0131] Determine whether a specific gesture for ending the recognition process is input. If not, continue to record the above information. If yes, serialize and normalize the gesture information according to the gesture type.

[0132] Select a specific function to bind and store the gesture sequence information in the gesture library.

[0133] The above-mentioned serialization and normalization of gesture information according to gesture type includes:

[0134] (2) Gesture Sequencing:

[0135] Read finger joint position information, wrist joint position information, time information and gesture type;

[0136] Determine whether it is a static gesture. If so, perform static gesture serialization; if not, perform dynamic gesture serialization and perform temporal domain normalization and spatial normalization of gesture information.

[0137] Static gesture serialization includes: obtaining the three-dimensional coordinates of the wrist W, thumb T, index finger I, middle finger M, ring finger R and pinky finger P of the static gesture, and marking the three-dimensional coordinates of the above feature points of the static gesture as and Where D represents the current gesture, and the right subscripts W, T, I, M, R, and P represent the feature points corresponding to the wrist to the little finger. The above feature point pairs are combined in pairs to obtain 15 feature point pairs, and the feature point distance of each feature point pair is calculated. For example, the feature point pair and The distance is:

[0138]

[0139] in, represents the three-dimensional coordinates of the wrist, represents the three-dimensional coordinates of the index finger, Used to indicate the current static gesture and The distance between feature points.

[0140] By calculating the feature point distances of the plurality of feature point pairs and arranging them in this way, a first serialized feature can be obtained.

[0141] The dynamic gesture serialization includes: selecting the index finger I of the dynamic gesture as the motion feature point, obtaining the three-dimensional coordinates of the index finger I at multiple sampling moments, and marking the three-dimensional coordinates of the index finger I at the i-th moment as Where D represents the current gesture, I represents the index finger, and t i represents the i-th moment, i>1; then, in the adjacent moments from the i-1th moment to the i-th moment, the displacement of the feature point of the index finger I is Thus, we can preliminarily obtain the serialization characteristics of index finger I.

[0142] The above-mentioned normalization of gesture information in the time domain includes:

[0143] Normalize the sampling time of dynamic gestures:

[0144]

[0145] in, represents the normalized sampling time of the i-th moment, t i represents the actual sampling time, t s Indicates the start time of the action, t e Indicates the end time of the action.

[0146] Normalize the sampling interval of dynamic gestures:

[0147] Δt norm =1 / N

[0148] Among them, Δt norm represents the normalized sampling interval, and N represents the total number of samples of dynamic gestures.

[0149] The displacement of the feature point of the index finger I at each adjacent moment is calculated based on the normalized sampling moment and the total number of samples:

[0150]

[0151] in, represents the displacement of the feature point of the index finger I in the adjacent moments from the normalized i-1th moment to the i-th moment, represents the normalized three-dimensional coordinate of the index finger I at the i-th moment, Represents the normalized three-dimensional coordinates of the index finger I at the i-1th moment.

[0152] The above-mentioned normalization of the gesture information space domain includes:

[0153] Based on the serialized features normalized in the time domain, the normalized displacement of the feature point of the index finger I at each moment is calculated:

[0154]

[0155] in, represents the actual displacement distance of the index finger I from the i-1th moment to the i-th moment, Represents the normalized displacement distance of the index finger I from the i-1th moment to the i-th moment.

[0156] The serialized features of the current gesture are determined based on the normalized displacement of feature points at multiple moments.

[0157] (3) Static gesture and dynamic gesture recognition

[0158] Detect the special gesture for starting gesture recognition, determine whether the specific starting gesture is input, if so, proceed to the next step; if not, continue to determine whether the specific starting gesture is input.

[0159] Record finger joints, wrist position information, time information and gesture type information.

[0160] Determine whether a specific gesture to end the recognition process is input, if so, proceed to the next step; if not, continue to record the relevant information of the previous step.

[0161] The gesture information is serialized and normalized according to the gesture type. The serialization and normalization can refer to the above (2) gesture serialization steps.

[0162] The gesture is compared with the preset gestures stored in the gesture library, and a comparison score is calculated.

[0163] Determine whether there is a gesture in the gesture library with a comparison score higher than the threshold. If so, find the gesture with the highest score among the gestures higher than the threshold and enter (4) error feedback. If not, prompt that the gesture is not included yet.

[0164] The above-mentioned comparing the gesture with the preset gesture stored in the gesture library and calculating the comparison score includes:

[0165] If the gesture type of the current gesture is a static gesture, then:

[0166] The current gesture is compared with multiple preset gestures in sequence. In the comparison with a single preset gesture, the following can be obtained: Figure 6 The serialized features of the current gesture - preset gesture are shown, where L represents the feature point distance of each feature point pair, D represents the current gesture, S represents the preset gesture, W, T, I, M, R and P represent the feature points of the wrist, thumb, index finger, middle finger, ring finger and little finger respectively, and each feature point pair in the serialized features of the current gesture corresponds to the feature point pair in the serialized features of the preset gesture.

[0167] The distance between the feature points in the current gesture serialization feature and the corresponding feature points in the preset gesture serialization feature are input into G(x,y) respectively:

[0168]

[0169] Among them, |xy| is the absolute value of the distance difference of the feature points, and ΔL1, ΔL2, ΔL3, ΔL4, ΔL5 and ΔL6 represent different preset distance difference standards, that is, different scores are assigned according to different distance difference ranges.

[0170] The total score G of the current gesture and the preset gesture is calculated based on the score of the distance difference of each feature point. score :

[0171]

[0172] Based on the total scores of all preset gestures, the preset gesture with the highest total score can be considered as the optimal solution for gesture recognition.

[0173] If the gesture type of the current gesture is a dynamic gesture, then:

[0174] The current gesture is compared with multiple preset gestures in sequence. In the comparison with a single preset gesture, the following can be obtained: Figure 7 The serialized features of the current gesture and the preset gesture are shown, where P represents the displacement of the feature points at each adjacent moment, D represents the current gesture, S represents the preset gesture, I represents the index finger, norm represents the normalized value, s represents the start time of the action, e represents the end time of the action, i represents the i-th moment, and each moment in the serialized features of the current gesture corresponds to a moment in the serialized features of the preset gesture. The scores of the current gesture and the preset gesture can refer to the score calculation method of the static gesture mentioned above.

[0175] (4) Error feedback

[0176] Determine whether the highest score is unique. If so, find the function bound to the gesture and execute it;

[0177] If not, find all the functions bound to the gestures with the highest scores and prompt the user to manually select them on the front end; the user clicks a gesture option box; based on the gesture selected by the user, find the function bound to the gesture and execute it.

[0178] Furthermore, the gesture recognition method of this embodiment, after successful gesture recognition, in addition to realizing the function bound to the gesture, can also use gestures or air drawing as digital input for the text box under the premise of activating the text box. For example, extending the index finger means "1", extending the index finger and middle finger means "2", or drawing "1" in the air to realize gesture input.

[0179] Furthermore, considering that a single gesture is too simple, a set of actions may require the user to spend more effort to complete. The gesture recognition method of this embodiment may also include recognition of combined gestures, such as Figure 5 As shown, the details are as follows:

[0180] When gesture recognition starts, it is determined whether the combination gesture is turned on. Whether the combination gesture is turned on can be based on the user's setting of the special gesture to indicate turning on or off the combination gesture recognition;

[0181] If the judgment is yes, that is, the combination gesture is turned on, the combination gesture sampling is performed; it is determined whether a special gesture indicating the end of the gesture is recognized, if not, the sampling continues, and if so, the gesture ends, and multi-gesture recognition is performed based on the sampled data.

[0182] If the judgment is no, the combined gesture is not enabled and single gesture recognition is performed.

[0183] Determine whether the recognition is successful or not. If so, execute the function bound to the recognized gesture.

[0184] The gesture recognition method of this embodiment can also be applied to Figure 4 In the gesture recognition device shown, the gesture recognition device includes: an image capturing module, an image processing module, a gesture information serialization module and a sequence comparison and scoring module.

[0185] Among them, the image capture module is used to take pictures using a depth camera and transmit the photo data to the image processing module.

[0186] The image processing module is used to identify hand features based on photo data and determine whether the image contains hand features. If so, it proceeds to the next step; if not, it returns to the image processing module to continue collecting; extract the pixel coordinates of the finger joints and wrist joints; reset the finger joint coordinates with the wrist joint coordinates as the origin; and send the finger joint point coordinates to the gesture information serialization module for serialization processing.

[0187] The gesture information serialization module is used to determine the gesture type based on the coordinates of the finger joint points and whether it is a static gesture. If so, static gesture serialization is performed; if not, dynamic gesture serialization is performed; and the sequence is sent to the sequence comparison and scoring module.

[0188] The sequence comparison and scoring module is used to compare and score the sequence with the sequence in the gesture library; determine whether there is a gesture in the gesture library with a comparison score higher than the threshold; if not, prompt an error; if so, determine whether there are multiple highest comparison scores, if so, prompt the user to make a selection, and execute the software function corresponding to the gesture based on the user's selection; if not, execute the software function corresponding to the gesture.

[0189] The gesture recognition method provided in this embodiment is a gesture recognition method that does not require neural network or machine learning training and recognition. It can match the current gesture with the preset gesture based on the processing of features such as fingers and wrists to obtain gesture recognition results, which can reduce recognition time and the amount of data calculation, and achieve the effect of improving gesture recognition speed and accuracy. By recognizing the pixel coordinates of the fingers and wrists based on the depth image, and then obtaining the three-dimensional coordinates of the fingers and wrists based on the pixel coordinates, the hand features can be recognized. After the hand features are recognized, the three-dimensional coordinates are calibrated with the three-dimensional coordinates of the wrist as the origin, and the situation that the gesture recognition is inaccurate due to the overall position offset of the gesture can be avoided, thereby achieving the effect of improving the gesture recognition accuracy. Calculating the relative positions between multiple feature points of the fingers and wrists can avoid the problem of low recognition rate caused by the overall offset of the three-dimensional coordinates. Based on the uniqueness between the relative distance feature and the gesture, the relative distance can better represent the gesture itself and be used for matching calculation, which can achieve the effect of improving the gesture recognition accuracy. Calculating the matching degree of each pair of corresponding feature points, thereby calculating the gesture matching degree of the entire gesture, can achieve the effect of improving the gesture recognition accuracy. By obtaining the three-dimensional coordinates of any feature point of a dynamic gesture at multiple moments, and obtaining the first serialized feature based on the displacement of the feature point at multiple adjacent moments, the motion capture of dynamic gestures can be achieved. The dynamic gestures are normalized in time and space to unify the serialized feature specifications of the current gesture and the preset gesture, so as to avoid the influence of the inconsistency of the duration or amplitude of the dynamic gesture on the gesture recognition, thereby improving the accuracy of gesture recognition. By calculating the difference between the displacement of each feature point in the current gesture serialization feature and the preset gesture serialization feature, the matching degree of each pair of corresponding feature point displacements can be calculated, thereby calculating the gesture matching degree of the entire dynamic gesture, thereby improving the accuracy of gesture recognition.

[0190] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0191] Based on the same inventive concept, the embodiment of the present application also provides a gesture recognition device for implementing the gesture recognition method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more gesture recognition device embodiments provided below can refer to the limitations on the gesture recognition method above, and will not be repeated here.

[0192] In one embodiment, Figure 8 As shown, a gesture recognition device is provided, comprising: an acquisition module 100, a feature module 200, a matching module 300 and a determination module 400, wherein:

[0193] An acquisition module 100 is used to acquire gesture space information of a current gesture based on a received gesture recognition instruction; the gesture space information includes three-dimensional coordinates of fingers and wrists;

[0194] A feature module 200, configured to obtain a first serialized feature of the current gesture based on the three-dimensional coordinates of the finger and the wrist;

[0195] A matching module 300, configured to sequentially calculate the gesture matching degree of the first serialized feature and the second serialized features of a plurality of preset gestures;

[0196] The determination module 400 is used to determine the gesture recognition result of the current gesture based on the gesture matching degree.

[0197] In one of the embodiments, the acquisition module 100 is also used to: acquire a depth image captured by an image acquisition device based on a received gesture recognition instruction; recognize hand features of the current gesture based on the depth image, the hand features including pixel coordinates of fingers and wrists; and determine the three-dimensional coordinates of the fingers and wrist based on the pixel coordinates.

[0198] In one embodiment, the gesture includes a static gesture, and the feature module 200 is further used to: determine multiple feature point pairs between the finger and the wrist based on the three-dimensional coordinates of the finger and the wrist; calculate the feature point distances of the multiple feature point pairs; and determine the first serialized feature of the static gesture based on the multiple feature point distances.

[0199] In one embodiment, the second serialized feature includes a second serialized feature of a preset static gesture, and the matching module 300 is further used to: determine feature point distance differences of multiple groups of feature point pairs based on the first serialized feature and the second serialized feature of the preset static gesture; calculate feature score values ​​of multiple groups of feature point pairs based on the feature point distance differences; and obtain the gesture matching degree according to the feature score values ​​of the multiple feature point pairs.

[0200] In one embodiment, the gesture includes a dynamic gesture, and the feature module 200 is further used to: take any finger or wrist as a feature point, and obtain the three-dimensional coordinates of the feature point at multiple moments; based on the three-dimensional coordinates of the feature point at multiple moments, calculate the feature point displacement of the feature point in multiple adjacent moments; based on the feature point displacement, determine the first serialized feature of the dynamic gesture.

[0201] In one embodiment, the feature module 200 is also used to: calculate the normalized sampling time and normalized sampling interval of the dynamic gesture in multiple adjacent moments based on the action start time, action end time and initial sampling interval of the adjacent moments of the dynamic gesture; calculate the normalized displacement of the feature points in the multiple adjacent moments based on the normalized sampling time, normalized sampling interval, the feature point displacement in the adjacent moments, and the total displacement of the feature points of the dynamic gesture; obtain the first serialized feature of the dynamic gesture based on the normalized displacement of the feature points in the multiple adjacent moments.

[0202] In one embodiment, the second serialized feature includes a second serialized feature of a preset dynamic gesture, and the matching module 300 is further used to: determine a feature point displacement difference of multiple feature point displacements based on the first serialized feature and the second serialized feature of the preset dynamic gesture; calculate feature score values ​​of the multiple feature point displacements based on the feature point displacement differences; and obtain the gesture matching degree according to the feature score values ​​of the multiple feature point displacements.

[0203] Each module in the above-mentioned gesture recognition device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module above.

[0204] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a gesture recognition method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0205] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0206] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the gesture recognition method of any of the above embodiments is implemented:

[0207] Acquire gesture space information of the current gesture based on the received gesture recognition instruction; the gesture space information includes three-dimensional coordinates of the fingers and wrist;

[0208] Based on the three-dimensional coordinates of the finger and the wrist, obtain a first serialized feature of the current gesture;

[0209] sequentially calculating gesture matching degrees of the first serialized feature and second serialized features of a plurality of preset gestures;

[0210] A gesture recognition result of the current gesture is determined based on the gesture matching degree.

[0211] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the gesture recognition method of any of the above embodiments is implemented:

[0212] Acquire gesture space information of the current gesture based on the received gesture recognition instruction; the gesture space information includes three-dimensional coordinates of the fingers and wrist;

[0213] Based on the three-dimensional coordinates of the finger and the wrist, obtain a first serialized feature of the current gesture;

[0214] sequentially calculating gesture matching degrees of the first serialized feature and second serialized features of a plurality of preset gestures;

[0215] A gesture recognition result of the current gesture is determined based on the gesture matching degree.

[0216] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0217] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0218] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0219] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A gesture recognition method, characterized in that: The method comprises: Acquire gesture space information of the current gesture based on the received gesture recognition instruction; the gesture space information includes three-dimensional coordinates of the fingers and wrist; Based on the three-dimensional coordinates of the finger and the wrist, obtain a first serialized feature of the current gesture; sequentially calculating gesture matching degrees of the first serialized feature and second serialized features of a plurality of preset gestures; A gesture recognition result of the current gesture is determined based on the gesture matching degree.

2. The method according to claim 1, characterized in that The acquiring of gesture space information of the current gesture based on the received gesture recognition instruction includes: Based on the received gesture recognition instruction, acquiring a depth image captured by an image acquisition device; Recognize hand features of the current gesture based on the depth image, the hand features including pixel coordinates of fingers and wrists; Based on the pixel coordinates, the three-dimensional coordinates of the fingers and wrist are determined.

3. The method according to claim 1, characterized in that The gesture includes a static gesture, and the first serialized feature of the current gesture obtained based on the three-dimensional coordinates of the finger and the wrist includes: Based on the three-dimensional coordinates of the finger and the wrist, determining a plurality of feature point pairs between the finger and the wrist; Calculating feature point distances of a plurality of feature point pairs; Based on the multiple feature point distances, a first serialized feature of the static gesture is determined.

4. The method according to claim 3, characterized in that The second serialization feature includes a second serialization feature of a preset static gesture, and the sequentially calculating the gesture matching degree between the first serialization feature and the second serialization features of a plurality of preset gestures includes: Determining feature point distance differences of multiple groups of feature point pairs based on the first serialized feature and the second serialized feature of the preset static gesture; Based on the feature point distance differences, calculating feature score values ​​of multiple groups of feature point pairs; The gesture matching degree is obtained according to the feature score values ​​of the plurality of feature point pairs.

5. The method according to claim 1, characterized in that The gesture includes a dynamic gesture, and the first serialized feature of the current gesture obtained based on the three-dimensional coordinates of the finger and the wrist includes: Taking any finger or wrist as a feature point, obtaining the three-dimensional coordinates of the feature point at multiple times; Based on the three-dimensional coordinates of the feature point at multiple moments, calculating the feature point displacement of the feature point within multiple adjacent moments; Based on the feature point displacement, a first serialized feature of the dynamic gesture is determined.

6. The method according to claim 5, characterized in that The determining the first serialized feature of the dynamic gesture based on the feature point displacement further includes: Calculate the normalized sampling time and the normalized sampling interval of the dynamic gesture within a plurality of adjacent moments based on the action start time and the action end time of the dynamic gesture and the initial sampling interval of the adjacent moments; Calculating the normalized displacements of the feature points in the plurality of adjacent moments based on the normalized sampling moment, the normalized sampling interval, the displacements of the feature points in the adjacent moments, and the total displacements of the feature points of the dynamic gesture; Based on the normalized displacement of the feature points in the plurality of adjacent moments, a first serialized feature of the dynamic gesture is obtained.

7. The method according to claim 5, characterized in that The second serialization feature includes a second serialization feature of a preset dynamic gesture, and the sequentially calculating the gesture matching degree between the first serialization feature and the second serialization features of a plurality of preset gestures includes: Determine a feature point displacement difference of a plurality of feature point displacements based on the first serialized feature and the second serialized feature of the preset dynamic gesture; Calculating feature score values ​​of the displacements of a plurality of feature points based on the feature point displacement differences; The gesture matching degree is obtained according to the feature score values ​​of the displacements of the plurality of feature points.

8. A gesture recognition device, characterized in that: The device comprises: An acquisition module, configured to acquire gesture space information of a current gesture based on a received gesture recognition instruction; the gesture space information includes three-dimensional coordinates of fingers and wrists; A feature module, used for obtaining a first serialized feature of the current gesture based on the three-dimensional coordinates of the finger and the wrist; A matching module, used for sequentially calculating the gesture matching degree of the first serialized feature and the second serialized features of a plurality of preset gestures; A determination module is used to determine a gesture recognition result of the current gesture based on the gesture matching degree.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images