Template matching gesture recognition method, system, device and medium based on recommendation algorithm

Abstract

The present invention discloses a template matching gesture recognition method, system, device and medium based on a recommendation algorithm, the method comprising: obtaining three-dimensional hand motion data, performing normalization processing, defining a cumulative feature change index to determine the start and end state of a gesture; extracting key shape features and displacement features of the three-dimensional hand motion data, fusing and constructing a feature vector high-dimensional space model, and establishing a gesture template library; proposing a modified cosine similarity index, completing matching calculations based on feature vector space, and recommending the formation of a target gesture; according to user feedback, fusing the correctly recognized gesture with the current gesture template library, and realizing dynamic and adaptive updating of the gesture template library. Compared with traditional methods, the present invention greatly improves recognition accuracy and stability, can ensure the robustness of small sample data sets, and is beneficial to improving the usability and interactive experience of Leap Motion.

Description

Technical Field

[0001] The present invention relates to a template matching gesture recognition method, system, device and medium based on a recommendation algorithm, and belongs to the field of human-computer interaction gesture recognition. Background Art

[0002] In recent years, gesture interaction technology has developed rapidly and has been widely used in virtual reality scenarios in many fields such as education and entertainment, medical assistance, aerospace, etc. In order to improve the interactive experience, easy-to-operate, high-precision, and lightweight gesture recognition systems have become the focus of research and development.

[0003] According to the different ways of collecting gesture data, gesture recognition methods can be divided into two types. One type is based on wearable device technology, which analyzes and processes the raw gesture data captured by physical devices such as data gloves to achieve the goal of gesture recognition. The other type is based on computer vision technology, which uses one or more cameras to collect gesture images and uses visual processing technology to analyze the features of image sequences to complete gesture recognition. As the technology of upgrading ordinary cameras to real-time depth sensors becomes more and more mature, visual sensing devices represented by Leap Motion have promoted the application of three-dimensional gestures due to their low price and freedom of operation. Gesture recognition methods based on these portable devices have become a hot topic of research.

[0004] Leap Motion SDK provides three native gestures that can be used directly: Swipe, Circle, and Tap, but there are few related usage functions and limited recognition accuracy, which makes it difficult to fully meet the needs of virtual interaction. The gesture motion data captured in real time by Leap Motion has the characteristics of complex spatial information, high degree of freedom, randomness and disorder. Existing gesture recognition technology is difficult to fully and effectively express gesture characteristics, thus affecting the accurate and efficient recognition of gestures. In addition, the current gesture recognition method developed based on Leap Motion lacks consideration of individual differences among users, and fails to make full use of user historical data. It is difficult to meet the needs of users for personalized experience in practical applications. Especially for the interactive operation of CAE application software interfaces and three-dimensional images, it is necessary to develop richer, more friendly, efficient and easy-to-expand gesture recognition methods.

[0005] Currently, there are three main gesture recognition methods for visual sensing devices such as Leap Motion and Kinect: heuristic recognition method based on rule coding, recognition method based on machine learning, and template matching gesture recognition method based on small sample size learning.

[0006] Heuristic recognition methods use certain significant features of gestures to encode rules, and are mainly used to determine the position and direction of gestures. For example, when using the skeletal data obtained by Kinect to develop soldier movement training software, the action of making the head position exceed the preset height is defined as "jumping"; and in music performance interactive games, the movement acceleration of the game controller is used to control the performance of musical instruments. The encoding idea of ​​heuristic recognition is simple but difficult to modify. It is suitable for a small number of fixed gestures and has limited application scenarios.

[0007] Recognition methods based on machine learning have been widely studied and applied. Existing experiments have verified the gesture recognition capabilities based on Leap Motion and radial basis function (RBF) neural network models; the motion data optimization method based on convolutional neural network (CNN) solves the self-occlusion problem of Leap Motion devices; in addition, there is an optimized multi-class support vector classifier (multiclass-SVC) to recognize static gestures, and the accuracy of gesture recognition is improved by iteratively increasing the sample capacity. Gesture recognition methods based on machine learning focus on the classification of gesture postures rather than the feature extraction of hand movements, which requires a large amount of data samples and is relatively costly.

[0008] Template matching recognition methods based on small sample sizes have received research attention in recent years due to their low cost. Template matching recognition of static two-handed gestures is achieved through the nearest neighbor algorithm, and interference gestures are screened using similarity judgment; the original input trajectory is resampled and scaled, and three-dimensional gesture matching is achieved by calculating the optimal rotation matrix and minimum Euclidean distance between the input gesture and the template gesture; there is also a static and dynamic gesture recognition method based on spatial fuzzy matching, which performs feature point matching on the three-dimensional coordinate data of the 16 joints obtained by Leap Motion, but the redundancy of the relevant gesture feature model is high and the amount of calculation is large. The gesture recognition method based on template matching has a small demand for sample data and only requires a small number of inherent templates for recognition. However, gesture movements are dynamic sequences with temporal, spatial and individual differences, and a small number of inherent templates cannot meet the needs of high-precision gesture recognition. Summary of the invention

[0009] In view of the problems of complex existing template matching algorithms, redundant model feature expression, lack of dynamic scalability, etc., the present invention provides a template matching gesture recognition method, system, computer device and storage medium based on a recommendation algorithm, which can dynamically expand and adaptively update the template library, provide users with high-precision, lightweight and personalized gesture recognition services, and help improve the usability and interactive experience of Leap Motion.

[0010] The first object of the present invention is to provide a template matching gesture recognition method based on a recommendation algorithm.

[0011] A second object of the present invention is to provide a template matching gesture recognition system based on a recommendation algorithm.

[0012] A third object of the present invention is to provide a computer device.

[0013] A fourth object of the present invention is to provide a storage medium.

[0014] The first object of the present invention can be achieved by adopting the following technical solutions:

[0015] A template matching gesture recognition method based on a recommendation algorithm, the method comprising:

[0016] Obtain the 3D hand motion data collected by Leap Motion, normalize the 3D hand motion data, and define the cumulative feature change index to determine the start and end state of the gesture;

[0017] Extract key shape features and displacement features of 3D hand motion data to fully characterize gesture information;

[0018] Fuse key shape features and displacement features, construct a high-dimensional space model of feature vectors, and establish a gesture template library;

[0019] Based on the matching algorithm of feature vector space, the modified cosine similarity between the gesture to be recognized in each frame and the template gesture in the gesture template library is calculated as the unit matching rate;

[0020] According to the unit matching rate, the average matching rate between the gestures to be recognized in all frames and the template gestures in the gesture template library is calculated, and the corresponding category names of the template gestures with an average matching rate higher than the gesture similarity threshold are added to the recommendation list, sorted by the size of the average matching rate, and the highest template gesture category in the recommendation list is output as the result of gesture recognition;

[0021] According to user feedback, the correctly recognized gestures are integrated with the current gesture template library to achieve dynamic and adaptive updating of the gesture template library.

[0022] Furthermore, the three-dimensional hand motion data is normalized as follows:

[0023]

[0024] Among them, x s represents the feature quantity after the maximum value is normalized, x represents the original data of the feature quantity, and x min Indicates the minimum value of the feature quantity within the range of variation, x max Indicates the maximum value of the feature quantity within the variation range.

[0025] Furthermore, the definition of the cumulative feature change index to determine the start and end state of the gesture specifically includes:

[0026] The calculation formula for the cumulative feature change is defined as follows:

[0027]

[0028] Among them, x i,j represents the jth feature value of the i-th frame, and n represents the number of feature values;

[0029] If the cumulative feature change amount R is greater than the feature change threshold, it is determined that the gesture has started; if the cumulative feature change amount R approaches zero, it is determined that the gesture has stopped.

[0030] Furthermore, the fusion of key shape features and displacement features, construction of a feature vector high-dimensional space model, and establishment of a gesture template library specifically include:

[0031] The key shape features and displacement features are connected in series and fused into a multi-dimensional feature vector of a single gesture in a certain frame. Each template gesture in the template gesture set is expressed as a corresponding feature vector, and a multi-dimensional feature vector space model is constructed to establish a gesture template library.

[0032] Furthermore, the matching algorithm based on the feature vector space calculates the modified cosine similarity between each frame of the gesture to be recognized and the template gesture in the gesture template library, specifically including:

[0033] In the multidimensional space, the feature vector F of each frame of gesture to be recognized is o,i Respectively with each template gesture feature vector F d,j The cosine similarity calculation is as follows:

[0034]

[0035] Among them, σ i,j represents the cosine similarity between the gesture to be recognized in the i-th frame and the feature vector of the j-th template gesture, f o,k represents the kth eigenvalue of the gesture to be recognized, f d,k Represents the kth feature value of the template gesture;

[0036] On the basis of cosine similarity, the lengths of two feature vectors are compared, and the cosine similarity is corrected by combining the direction and size of the feature vectors, as shown in the following formula:

[0037]

[0038] Among them, σ * i,j It represents the modified cosine similarity between the gesture to be recognized in the i-th frame and the feature vector of the j-th template gesture.

[0039] Furthermore, the average matching rate between the gestures to be recognized in all frames and the template gestures in the gesture template library is calculated according to the unit matching rate, and the corresponding category names of the template gestures with an average matching rate higher than the gesture similarity threshold are added to the recommendation list, specifically including:

[0040] According to the unit matching rate, the average matching rate between the gestures to be recognized in all frames and the template gestures in the gesture template library is calculated as follows:

[0041]

[0042] Where N is the number of frames of the gesture to be recognized;

[0043] When the average matching rate is higher than the gesture similarity threshold, the corresponding category name of the template gesture is added to the recommendation list until the gesture to be recognized and all template gestures complete the similarity calculation.

[0044] Furthermore, according to user feedback, the correctly recognized gesture is integrated with the current gesture template library to achieve dynamic and adaptive update of the gesture template library, which specifically includes:

[0045] The user's feedback on the gesture recognition results is obtained through human-computer interaction. If the gesture is correctly recognized and has a high similarity with the template gesture, the correctly recognized gesture is merged with the template gesture, and the current gesture template library is updated using the moving average method.

[0046] The second object of the present invention can be achieved by adopting the following technical solutions:

[0047] A template matching gesture recognition system based on a recommendation algorithm, the system comprising:

[0048] The gesture data acquisition and preprocessing module is used to obtain the three-dimensional hand motion data collected by Leap Motion, normalize the three-dimensional hand motion data, and define the cumulative feature change index to determine the start and end state of the gesture;

[0049] The gesture feature extraction module is used to extract the key shape features and displacement features of 3D hand motion data to fully characterize gesture information;

[0050] The gesture template library building module is used to fuse key shape features and displacement features, construct a high-dimensional space model of feature vectors, and build a gesture template library;

[0051] A gesture similarity calculation module is used to calculate the modified cosine similarity between the gesture to be recognized in each frame and the template gesture in the gesture template library based on the matching algorithm of the feature vector space as a unit matching rate;

[0052] The gesture recognition result output module is used to calculate the average matching rate between the gestures to be recognized in all frames and the template gestures in the gesture template library according to the unit matching rate, add the corresponding category names of the template gestures with an average matching rate higher than the gesture similarity threshold to the recommendation list, sort them according to the size of the average matching rate, and output the highest template gesture category in the recommendation list as the result of gesture recognition;

[0053] The gesture template library update module is used to merge the correctly recognized gestures with the current gesture template library for use in a new round of gesture recognition based on user feedback, thereby realizing dynamic and adaptive update of the gesture template library.

[0054] The third object of the present invention can be achieved by adopting the following technical solutions:

[0055] A computer device comprises a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the above-mentioned template matching gesture recognition method is implemented.

[0056] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0057] A storage medium stores a program, and when the program is executed by a processor, the above-mentioned template matching gesture recognition method is implemented.

[0058] The present invention has the following beneficial effects compared with the prior art:

[0059] The template matching gesture recognition method of the present invention is implemented based on the recommended algorithm of modified cosine similarity and updated template. Compared with the traditional gesture recognition method, it can greatly improve the recognition accuracy and stability, ensure the robustness of small sample data sets, realize dynamic expansion and adaptive update of the template library, and provide users with high-precision, lightweight, personalized gesture recognition services, which is beneficial to improving the usability and interactive experience of Leap Motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0061] Figure 1 This is a simplified flow chart of a template matching gesture recognition method based on a recommendation algorithm according to Embodiment 1 of the present invention.

[0062] Figure 2This is a detailed flow chart of a template matching gesture recognition method based on a recommendation algorithm according to Embodiment 1 of the present invention.

[0063] Figure 3 This is a diagram illustrating the gesture definition of Example 1 of the present invention.

[0064] Figure 4 This is a diagram illustrating gesture parameters and corresponding Leap Motion functions used in Example 1 of the present invention.

[0065] Figure 5 This is a comparison chart of the recognition accuracy of the template matching gesture recognition method based on the recommendation algorithm in Example 1 of the present invention and other gesture recognition methods.

[0066] Figure 6 This is a comparison chart of the recognition recall rate of the template matching gesture recognition method based on the recommendation algorithm in Example 1 of the present invention and other gesture recognition methods.

[0067] Figure 7 This is a comparison chart of the response time of the template matching gesture recognition method based on the recommendation algorithm in Example 1 of the present invention and other gesture recognition methods.

[0068] Figure 8 This is a gesture recognition confusion matrix diagram of the heuristic gesture recognition method according to Example 1 of the present invention.

[0069] Fig. 9 This is a gesture recognition confusion matrix diagram of the template matching gesture recognition method based on the recommendation algorithm in Example 1 of the present invention.

[0070] Fig.10 This is a structural block diagram of a template matching gesture recognition system based on a recommendation algorithm according to Embodiment 2 of the present invention.

[0071] Fig.11 This is a structural block diagram of a computer device according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0072] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0073] Embodiment 1:

[0074] This embodiment provides a template matching gesture recognition method based on a recommendation algorithm. The method can be used in a gesture interactive control system of a CAE application interface and a three-dimensional image. First, Leap Motion is used to collect and preprocess three-dimensional hand motion data, extract key shape features and displacement features, fuse and construct a high-dimensional space model of feature vectors, and establish a gesture template library. Then, for the user input gesture, the modified cosine similarity between the gesture and the template gesture is calculated, and a target gesture is formed based on a Top-1 recommendation strategy for subsequent interactive tasks. Finally, based on user feedback, the correctly recognized gesture is merged with the original gesture template library, and the template library is updated for a new round of gesture recognition.

[0075] like Figure 1 and Figure 2 As shown, the template matching gesture recognition method based on the recommendation algorithm of this embodiment includes the following steps:

[0076] S201, obtaining three-dimensional hand motion data collected by Leap Motion, normalizing the three-dimensional hand motion data, and defining a cumulative feature change index to determine the start and end states of a gesture.

[0077] This embodiment can predefine gestures according to specific application scenario requirements; specifically, according to the application scenario requirements of CAE application program interface and three-dimensional image, based on the gesture interaction design concept of simple operation and easy learning, two static gestures of "open" and "clench fist" and eight dynamic gestures of "click" and "position" are defined. The specific gesture action characteristics and interactive uses are as follows: Figure 3 shown.

[0078] After defining the gesture, this embodiment uses Leap Motion to capture gesture data. The specific gesture parameters and corresponding Leap Motion functions used are as follows: Figure 4 As shown, the extracted features are normalized as follows to eliminate the adverse effects of the amplitude differences of the feature vectors on the recognition results:

[0079]

[0080] Among them, x s represents the feature quantity after the maximum value is normalized, x represents the original data of the feature quantity, and x min Indicates the minimum value of the feature quantity within the range of variation, x max Indicates the maximum value of the feature quantity within the variation range.

[0081] Based on the gesture features defined in the subsequent steps, the changes in the gesture feature values ​​of the 15 frames before the current frame are accumulated to judge the state of the gesture. The calculation formula for the accumulated feature change R is as follows:

[0082]

[0083] Among them, x i,j represents the j-th feature value of the i-th frame, and n represents the number of feature values.

[0084] If the cumulative feature change R is greater than the feature change threshold (the feature change threshold is 2 in this embodiment), it is judged that the gesture has started; if the cumulative feature change R approaches zero (less than 0.5 in this embodiment), it is judged that the gesture has stopped.

[0085] S202, extracting key shape features and displacement features of the three-dimensional hand motion data to fully characterize the gesture information.

[0086] The static gesture defined in this embodiment is a single hand shape, which can be fully expressed by describing the shape of the hand, while the dynamic gesture is a collection of a series of static gestures, involving time and space context, so the change of spatial position needs to be described.

[0087] Figure 3 The gestures are mainly manifested in three single-hand forms, namely, five fingers open, five fingers closed, and extended index finger. They are obviously different in the degree of finger bending and the number of detected fingers. Therefore, the degree of finger bending s1, the strength of fingertip distance s2, the number of fingers s3, the palm curvature s4, and the distance between the thumb and other fingers s5 are used as shape features to describe the shape. The first four feature quantities can be directly obtained by Leap Motion, and the fingertip distance feature of the thumb and other fingers is calculated as follows:

[0088]

[0089] Among them, s5 is the distance between the thumb and the fingertips of other fingers, f0 is the three-dimensional coordinate of the thumb tip, and f i (i=1, 2, 3, 4) are the three-dimensional coordinates of the fingertips of other fingers.

[0090] The spatial position change of dynamic gestures is characterized by translation along the three directions of the coordinate system XYZ. Therefore, the three-dimensional coordinate changes p1, p2, and p3 of the palm are used as displacement features:

[0091] p1=h x -h 0,x

[0092] p2=h y -h 0,y (4)

[0093] p3=h z -h 0,z

[0094] Among them, p i(i=1,2,3) is the moving distance of the palm along each coordinate axis, h j (j=x, y, z) is the three-dimensional coordinate of the palm of the hand in the current frame, h 0,j (j=x, y, z) is the three-dimensional coordinate of the palm of the hand in the initial frame.

[0095] S203, integrating key shape features and displacement features, constructing a high-dimensional space model of feature vectors, and establishing a gesture template library.

[0096] This embodiment connects key shape features and displacement features in series to fuse them into a multi-dimensional feature vector of a single gesture in a certain frame, expresses each template gesture in the template gesture set as a corresponding feature vector, constructs a multi-dimensional feature vector space model, and thus establishes a gesture template library.

[0097] Specifically, the key shape features and displacement features are connected in series and fused into an 8-dimensional feature vector F of a single gesture frame:

[0098] F=[s1,s2,s3,s4,s5,p1,p2,p3] (5)

[0099] Each gesture to be recognized contains N frames of data (N=20 in this embodiment), and the gesture feature vector F in each frame o,i (i=1,2,…,N) is:

[0100] F o,i =[s 1,i ,s 2,i ,s 3,i ,s 4,i ,s 5,i ,p 1,i ,p 2,i ,p 3,i ] (6)

[0101] Correspondingly, the feature vector F of each template gesture in the template gesture set d,j (j=1, 2, ..., M, in this embodiment, M=10) is:

[0102] F d,j =[s 1,j ,s 2,j ,s 3,j ,s 4,j ,s 5,j ,p 1,j ,p 2,j ,p 3,j ] (7)

[0103] Based on the gesture features defined above, an 8-dimensional feature vector space model can be constructed, in which each feature dimension is independent of each other, and the number of dimensions changes accordingly with the number of feature quantities, thereby establishing a gesture template library.

[0104] S204 , based on the matching algorithm of the feature vector space, calculate the modified cosine similarity between the gesture to be recognized in each frame and the template gesture in the gesture template library as the unit matching rate.

[0105] In vector space operations, the cosine similarity method is usually used to evaluate the degree of angle deviation between vectors. The feature vector F of each frame of gesture to be recognized is converted into o,i Respectively with each template gesture feature vector F d,j Perform similarity calculation to determine gesture matching:

[0106]

[0107] Among them, σ i,j represents the cosine similarity between the gesture to be recognized in the i-th frame and the feature vector of the j-th template gesture, f o,k represents the kth eigenvalue of the gesture to be recognized, f d,k Represents the kth feature value of the template gesture.

[0108] Since the cosine similarity calculation method has the limitation of being insensitive to the size of the vector, and the value of the feature data has a great influence on the matching result, it is also necessary to compare the length deviation between the feature vectors; in order to further improve the accuracy and stability of gesture recognition, this embodiment proposes a modified cosine similarity calculation method, which compares the lengths of two vectors based on cosine similarity, and uses the direction and size of the comprehensive vector as an indicator for matching judgment:

[0109]

[0110] Among them, σ * i,j It represents the modified cosine similarity between the gesture to be recognized in the i-th frame and the feature vector of the j-th template gesture.

[0111] The modified cosine similarity between the gesture to be recognized in each frame and the jth template gesture feature vector is calculated. As a unit match rate.

[0112] S205. Calculate the average matching rate between the gestures to be recognized in all frames and the template gestures in the gesture template library based on the unit matching rate, add the corresponding category names of the template gestures whose average matching rates are higher than the gesture similarity threshold to the recommendation list, sort them according to the average matching rates, and output the highest-ranked template gesture category in the recommendation list as the result of gesture recognition.

[0113] In this embodiment, after calculating the unit matching rate between each frame of the gesture to be recognized and the template gesture in the gesture template library, the average matching rate between all frames (N frames) of the gesture to be recognized and the template gesture in the gesture template library is calculated according to the unit matching rate.

[0114]

[0115] When the average matching rate is higher than the gesture similarity threshold (the gesture similarity threshold is set to 0.6 in this embodiment), the corresponding category name of the template gesture is added to the recommendation list until the gesture to be recognized and all template gestures have completed the similarity calculation, and are sorted according to the size of the average matching rate, and the highest template gesture category in the recommendation list is output as the result of gesture recognition; if the average matching rate with all template gestures is lower than the gesture similarity threshold, it is regarded as an interference gesture, and no result is output for gesture recognition.

[0116] S206. According to user feedback, the correctly recognized gesture is integrated with the current gesture template library to achieve dynamic and adaptive updating of the gesture template library.

[0117] Specifically, after the gesture recognition result is output, the user's feedback on the gesture recognition result is obtained through human-computer interaction. If the gesture is correctly recognized and the similarity with the template gesture is large (greater than 0.8 in this embodiment), the correctly recognized gesture is merged with the template gesture, and the current gesture template library is updated using the moving average method for use in a new round of gesture recognition.

[0118] This embodiment conducts comparative analysis of different recognition methods such as Leap Motion native gesture recognition, heuristic recognition, cosine similarity matching, and template library update. In the same environment, the original gesture data is collected by Leap Motion, and the execution Figure 3 There are 150 groups of each gesture action, and a total of 1500 continuous hand movement data samples are obtained for gesture recognition experiments.

[0119] The precision and recall results of the five gesture recognition methods are shown in Figure 2. Figure 5 and Figure 6 As shown in the results, it can be seen that the recommended algorithm based on modified cosine similarity and initial template (the gesture recognition method of this embodiment) has a high average recognition accuracy and average recognition recall rate for all gestures. After 50 rounds of template updates, the average recognition accuracy and average recognition recall rate are further improved. In addition, the modified cosine similarity calculation method has greatly improved the recognition effect of static gestures and dynamic gestures, and the recognition stability is enhanced.

[0120] Figure 7The response time of five gesture recognition methods is shown. The response time based on the recommended algorithm design is longer than that of the heuristic recognition, and the calculation amount is slightly larger, but it still has good real-time performance.

[0121] Figure 8 and Fig. 9 The gesture recognition confusion matrices of the heuristic recognition method, the recommended algorithm based on modified cosine similarity and the updated template are shown respectively; it can be seen from the results that the confusion rate of gesture recognition based on the recommended algorithm is controlled below 0.02, and the recognition stability is high, while the confusion rate of the heuristic recognition method has large fluctuations. Therefore, the recommended algorithm has obvious advantages in distinguishing similar gestures in continuous actions and eliminating interfering gestures.

[0122] It should be noted that although the method operations of the above embodiments are described in a specific order, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0123] Embodiment 2:

[0124] like Fig.10 As shown, this embodiment provides a template matching gesture recognition system based on a recommendation algorithm, which includes a gesture data acquisition and preprocessing module 1001, a gesture feature extraction module 1002, a gesture template library establishment module 1003, a gesture similarity calculation module 1004, a gesture recognition result output module 1005 and a gesture template library update module 1006. The specific functions of each module are as follows:

[0125] The gesture data acquisition and preprocessing module 1001 is used to acquire the three-dimensional hand motion data collected by Leap Motion, normalize the three-dimensional hand motion data, and define the cumulative feature change index to determine the start and end state of the gesture;

[0126] The gesture feature extraction module 1002 is used to extract key shape features and displacement features of the three-dimensional hand motion data to fully characterize the gesture information;

[0127] A gesture template library building module 1003 is used to fuse key shape features and displacement features, construct a feature vector high-dimensional space model, and build a gesture template library;

[0128] The gesture similarity calculation module 1004 is used to calculate the modified cosine similarity between the gesture to be recognized in each frame and the template gesture in the gesture template library based on the matching algorithm of the feature vector space as a unit matching rate;

[0129] The gesture recognition result output module 1005 is used to calculate the average matching rate between the gestures to be recognized in all frames and the template gestures in the gesture template library according to the unit matching rate, add the corresponding category names of the template gestures with an average matching rate higher than the gesture similarity threshold to the recommendation list, sort them according to the size of the average matching rate, and output the highest template gesture category in the recommendation list as the result of gesture recognition;

[0130] The gesture template library updating module 1006 is used to merge the correctly recognized gesture with the current gesture template library for use in a new round of gesture recognition based on user feedback, thereby realizing dynamic and adaptive updating of the gesture template library.

[0131] It should be noted that the system provided in this embodiment is only illustrated by the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0132] Embodiment 3:

[0133] This embodiment provides a computer device, such as Fig.11 As shown, it includes a processor 1102, a memory, an input device 1103, a display device 1104 and a network interface 1105 connected through a system bus 1101. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium 1106 and an internal memory 1107. The non-volatile storage medium 1106 stores an operating system, a computer program and a database. The internal memory 1107 provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the processor 1102 executes the computer program stored in the memory, the template matching gesture recognition method of the above-mentioned embodiment 1 is implemented as follows:

[0134] The three-dimensional hand motion data collected by Leap Motion is obtained, and the three-dimensional hand motion data is normalized, and the cumulative feature change index is defined to determine the start and end state of the gesture; the key shape features and displacement features of the three-dimensional hand motion data are extracted to fully characterize the gesture information; the key shape features and displacement features are fused to construct a feature vector high-dimensional space model, and a gesture template library is established; based on the matching algorithm of the feature vector space, the modified cosine similarity between the gesture to be recognized in each frame and the template gesture in the gesture template library is calculated as the unit matching rate; according to the unit matching rate, the average matching rate between the gesture to be recognized in all frames and the template gesture in the gesture template library is calculated, and the corresponding category names of the template gestures with an average matching rate higher than the gesture similarity threshold are added to the recommendation list, sorted according to the size of the average matching rate, and the highest template gesture category in the recommendation list is output as the result of gesture recognition; according to user feedback, the correctly recognized gesture is merged with the current gesture template library to realize dynamic and adaptive updating of the gesture template library.

[0135] Embodiment 4:

[0136] This embodiment provides a storage medium, which is a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the template matching gesture recognition method of the above embodiment 1 is implemented as follows:

[0137] The three-dimensional hand motion data collected by Leap Motion is obtained, and the three-dimensional hand motion data is normalized, and the cumulative feature change index is defined to determine the start and end state of the gesture; the key shape features and displacement features of the three-dimensional hand motion data are extracted to fully characterize the gesture information; the key shape features and displacement features are fused to construct a feature vector high-dimensional space model, and a gesture template library is established; based on the matching algorithm of the feature vector space, the modified cosine similarity between the gesture to be recognized in each frame and the template gesture in the gesture template library is calculated as the unit matching rate; according to the unit matching rate, the average matching rate between the gesture to be recognized in all frames and the template gesture in the gesture template library is calculated, and the corresponding category names of the template gestures with an average matching rate higher than the gesture similarity threshold are added to the recommendation list, sorted according to the size of the average matching rate, and the highest template gesture category in the recommendation list is output as the result of gesture recognition; according to user feedback, the correctly recognized gesture is merged with the current gesture template library to realize dynamic and adaptive updating of the gesture template library.

[0138] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0139] In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device, or device. In this embodiment, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program used by or in combination with an instruction execution system, device, or device. The computer program contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0140] The computer readable storage medium can be written in one or more programming languages ​​or a combination thereof to execute the computer program of the present embodiment, and the programming language includes an object-oriented programming language, such as Java, Python, C++, and also includes a conventional procedural programming language, such as C language or a similar programming language. The program can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0141] In summary, the template matching gesture recognition method of the present invention is implemented based on the recommended algorithm of modified cosine similarity and updated template. Compared with the traditional gesture recognition method, it can greatly improve the recognition accuracy and stability, ensure the robustness of small sample data sets, and realize dynamic expansion and adaptive update of the template library, providing users with high-precision, lightweight, personalized gesture recognition services, which is beneficial to improving the usability and interactive experience of Leap Motion.

[0142] The above description is only a preferred embodiment of the present invention. It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention.

Claims

1. A template matching gesture recognition method based on a recommendation algorithm, characterized in that: The method comprises: Obtain the 3D hand motion data collected by Leap Motion, normalize the 3D hand motion data, and define the cumulative feature change index to determine the start and end state of the gesture; Extract key shape features and displacement features of 3D hand motion data to fully characterize gesture information; Fuse key shape features and displacement features, construct a high-dimensional space model of feature vectors, and establish a gesture template library; Based on the matching algorithm of feature vector space, the modified cosine similarity between the gesture to be recognized in each frame and the template gesture in the gesture template library is calculated as the unit matching rate; According to the unit matching rate, the average matching rate between the gestures to be recognized in all frames and the template gestures in the gesture template library is calculated, and the corresponding category names of the template gestures with an average matching rate higher than the gesture similarity threshold are added to the recommendation list, sorted by the size of the average matching rate, and the highest template gesture category in the recommendation list is output as the result of gesture recognition; According to user feedback, the correctly recognized gestures are integrated with the current gesture template library to achieve dynamic and adaptive update of the gesture template library; The definition of the cumulative feature change index to determine the start and end state of the gesture specifically includes: The calculation formula for the cumulative feature change is defined as follows: Among them, x i,j represents the jth feature value of the i-th frame, and n represents the number of feature values; If the cumulative feature change R is greater than the feature change threshold, it is judged that the gesture has started; if the cumulative feature change R approaches zero, it is judged that the gesture has stopped; The matching algorithm based on the feature vector space calculates the modified cosine similarity between each frame of the gesture to be recognized and the template gesture in the gesture template library, as shown in the following formula: Among them, σ * i,j represents the modified cosine similarity between the gesture to be recognized in the i-th frame and the feature vector of the j-th template gesture, F o,i The feature vector representing the gesture to be recognized in each frame, F d,j Represents the feature vector of each template gesture, f o,k represents the kth eigenvalue of the gesture to be recognized, f d,k Represents the kth feature value of the template gesture; According to user feedback, the correctly recognized gesture is integrated with the current gesture template library to realize dynamic and adaptive update of the gesture template library, which specifically includes: The user's feedback on the gesture recognition results is obtained through human-computer interaction. If the gesture is correctly recognized and the similarity with the template gesture is greater than 0.8, the correctly recognized gesture is merged with the template gesture, and the current gesture template library is updated using the moving average method.

2. The template matching gesture recognition method according to claim 1, characterized in that: The three-dimensional hand motion data is normalized as follows: Among them, x s represents the feature quantity after the maximum value is normalized, x represents the original data of the feature quantity, and x min Indicates the minimum value of the feature quantity within the range of variation, x max Indicates the maximum value of the feature quantity within the variation range.

3. The template matching gesture recognition method according to claim 1, characterized in that: The method of fusing key shape features and displacement features, constructing a high-dimensional space model of feature vectors, and establishing a gesture template library specifically includes: The key shape features and displacement features are connected in series and fused into a multi-dimensional feature vector F for a single gesture frame: F=[s1,s2,s3,s4,s5,p1,p2,p3] Each gesture to be recognized contains N frames of data, and the gesture feature vector F in each frame o,i for: F o,i =[s 1,i ,s 2,i ,s 3,i ,s 4,i ,s 5,i ,p 1,i ,p 2,i ,p 3,i ] Correspondingly, the feature vector F of each template gesture in the template gesture set d,j for: F d,j =[s 1,j ,s 2,j ,s 3,j ,s 4,j ,s 5,j ,p 1,j ,p 2,j ,p 3,j ] A multi-dimensional feature vector space model is constructed to establish a gesture template library.

4. The template matching gesture recognition method according to claim 1, characterized in that: The method of calculating the average matching rate between the gestures to be recognized in all frames and the template gestures in the gesture template library according to the unit matching rate, and adding the corresponding category names of the template gestures with an average matching rate higher than the gesture similarity threshold to the recommendation list specifically includes: According to the unit matching rate, the average matching rate between the gestures to be recognized in all frames and the template gestures in the gesture template library is calculated as follows: Where N is the number of frames of the gesture to be recognized; When the average matching rate is higher than the gesture similarity threshold, the corresponding category name of the template gesture is added to the recommendation list until the gesture to be recognized and all template gestures complete the similarity calculation.

5. A template matching gesture recognition system based on a recommendation algorithm, characterized in that: The system comprises: The gesture data acquisition and preprocessing module is used to obtain the three-dimensional hand motion data collected by Leap Motion, normalize the three-dimensional hand motion data, and define the cumulative feature change index to determine the start and end state of the gesture; The gesture feature extraction module is used to extract the key shape features and displacement features of 3D hand motion data to fully characterize gesture information; The gesture template library building module is used to fuse key shape features and displacement features, construct a high-dimensional space model of feature vectors, and build a gesture template library; A gesture similarity calculation module is used to calculate the modified cosine similarity between the gesture to be recognized in each frame and the template gesture in the gesture template library based on the matching algorithm of the feature vector space as a unit matching rate; The gesture recognition result output module is used to calculate the average matching rate between the gestures to be recognized in all frames and the template gestures in the gesture template library according to the unit matching rate, add the corresponding category names of the template gestures with an average matching rate higher than the gesture similarity threshold to the recommendation list, sort them according to the size of the average matching rate, and output the highest template gesture category in the recommendation list as the result of gesture recognition; The gesture template library update module is used to merge the correctly recognized gestures with the current gesture template library for use in a new round of gesture recognition based on user feedback, thereby achieving dynamic and adaptive update of the gesture template library; The definition of the cumulative feature change index to determine the start and end state of the gesture specifically includes: The calculation formula for the cumulative feature change is defined as follows: Among them, x i,j represents the jth feature value of the i-th frame, and n represents the number of feature values; If the cumulative feature change R is greater than the feature change threshold, it is judged that the gesture has started; if the cumulative feature change R approaches zero, it is judged that the gesture has stopped; The matching algorithm based on the feature vector space calculates the modified cosine similarity between each frame of the gesture to be recognized and the template gesture in the gesture template library, as shown in the following formula: Among them, σ * i,j represents the modified cosine similarity between the gesture to be recognized in the i-th frame and the feature vector of the j-th template gesture, F o,i The feature vector representing the gesture to be recognized in each frame, F d,j Represents the feature vector of each template gesture, f o,k represents the kth eigenvalue of the gesture to be recognized, f d,k Represents the kth feature value of the template gesture; According to user feedback, the correctly recognized gesture is integrated with the current gesture template library to realize dynamic and adaptive update of the gesture template library, which specifically includes: The user's feedback on the gesture recognition results is obtained through human-computer interaction. If the gesture is correctly recognized and the similarity with the template gesture is greater than 0.8, the correctly recognized gesture is merged with the template gesture, and the current gesture template library is updated using the moving average method.

6. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the template matching gesture recognition method according to any one of claims 1 to 4 is implemented.

7. A storage medium storing a program, characterized in that: When the program is executed by a processor, the template matching gesture recognition method according to any one of claims 1 to 4 is implemented.

Images