VR element cosmic gesture recognition virtual interaction algorithm based on Leapmotion

Abstract

The invention discloses a Leapmotion-based virtual interaction algorithm for VR meta-universe gesture recognition, and particularly relates to the field of virtual interaction for VR meta-universe gesture recognition, and the algorithm comprises the steps: 1, data collection: employing a LeapMotion gesture controller to collect dynamic information of a hand of a user; step 2, feature extraction: carrying out preprocessing and feature extraction on the collected hand dynamic information; step 3, gesture recognition: inputting the extracted features into a gesture recognition algorithm, and judging the type of the user gesture through the trained classification model; step 4, virtual interaction: triggering a corresponding virtual interaction operation according to a classification result of the gestures; according to the method, a VR virtual interaction algorithm based on the LeapMotion is used, virtual reality interaction can be achieved, the LeapMotion can be used as a gesture controller of a virtual reality head-mounted display, a user can conduct interaction operation, grabbing, placing, dragging and the like through gestures, more visual and natural experience can be provided through the interaction mode, and the immersion feeling of the user is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of cranes, and more specifically, to a VR metaverse gesture recognition virtual interaction algorithm based on Leapmotion. Background Art

[0002] In traditional VR scenarios, users need to use handles to operate, which makes them less immersive and lacks a sense of involvement, making them lack an intuitive and natural experience and sense of immersion. Summary of the invention

[0003] The main technical problem solved by the present invention is to provide a VR metaverse gesture recognition virtual interaction algorithm based on Leapmotion, comprising the following steps:

[0004] Step 1: Data collection: Use the Leap Motion gesture controller to collect dynamic information of the user's hand;

[0005] Step 2: Feature extraction: preprocessing and feature extraction of the collected hand dynamic information;

[0006] Step 3: Gesture recognition: Input the extracted features into the gesture recognition algorithm and use the trained classification model to determine the type of user gesture;

[0007] Step 4: Virtual interaction: trigger corresponding virtual interaction operations based on the classification results of the gesture.

[0008] Furthermore, the user gesture in step one can capture the gesture by tracking the position of the hand, and the position formula can be expressed as: p = (x, y, z), where p represents the position of the hand and (x, y, z) represents the coordinates in three-dimensional space.

[0009] Furthermore, the speed of the user's finger in step one refers to the distance the hand moves in unit time, and the formula can be expressed as: v = (p2-p1) / t, where v represents the speed, p1 and p2 represent the hand positions at two time points, and t represents the time interval.

[0010] Furthermore, the acceleration of the user's finger in step one refers to the rate of change of the hand's speed per unit time, and its formula can be expressed as: a=(v2-v1) / t, wherein a represents acceleration, v1 and v2 represent speeds at two time points, and t represents the time interval.

[0011] Furthermore, the direction of the user's palm in step 2 refers to the orientation of the hand in three-dimensional space, and the formula can be expressed as: d = (dx, dy, dz), where d represents the direction vector and (dx, dy, dz) represents the direction component in three-dimensional space.

[0012] Furthermore, the feature extraction step in step 3 is as follows:

[0013] S1. Establishing a sample matrix according to the user hand data collected in step 1;

[0014] S2, using the sample matrix as the collected data of the non-smooth non-negative matrix decomposition algorithm, obtaining the objective function of the non-smooth non-negative matrix decomposition algorithm

[0015]

[0016]

[0017] X∈R m×n

[0018] Where V is the normalized sample matrix, V j Corresponding to the jth V value, X is the sample vector, X i Corresponding to the i-th X value, H is the feature representation matrix, W is the feature basis matrix, ‖‖ F represents the f-norm of the matrix, θ is a hyperparameter for adjusting the sparsity of the solution, θ∈[0,1], I represents the identity matrix, r is the dimension of the eigenvector, ‖ represents a vector of all 1s, ‖ T represents the transpose of the vector ‖, R represents the sample vector matrix, m and n represent the rows and columns in the sample vector respectively;

[0019] S3, constructing a proximal function according to the objective function, and constructing a proximal function according to the proximal function

[0020]

[0021] Find the optimal sample matrix, (where L is the Lipschitz constant, L = ||STWTWS||, <> represents the inner product of the matrix);

[0022] S4. Iterative formula for constructing a sample matrix based on the optimal sample matrix

[0023]

[0024] The sample matrix is ​​iteratively updated according to the iterative formula to obtain a feature matrix, where β 0 is a constant, giving a random initial value of iteration β 0 =1,Y 0 =H 0 , iteratively update according to the iterative formula, when the number of iterations reaches the set threshold k, the iteration is terminated, and H is obtained k That is the characteristic matrix, k represents the matrix product, Y kis an intermediate parameter, P() represents the projection gradient algorithm, and P(Z) represents projecting all negative numbers in the matrix Z to 0.

[0025] The beneficial effects of the VR metaverse gesture recognition virtual interaction algorithm based on Leapmotion of the present invention are:

[0026] The present invention uses a VR virtual interaction algorithm based on LeapMotion, which can realize virtual reality interaction: LeapMotion can be used as a gesture controller for a virtual reality head display, allowing users to perform interactive operations such as grabbing, placing, and dragging through gestures. This interactive mode can provide a more intuitive and natural experience and enhance the user's sense of immersion;

[0027] The present invention uses a sample matrix as the collected data of a non-smooth non-negative matrix decomposition algorithm, obtains the objective function of the non-smooth non-negative matrix decomposition algorithm, constructs a proximal function according to the objective function, obtains the optimal sample matrix according to the proximal function, constructs an iterative formula of the sample matrix according to the optimal sample matrix, iteratively updates the sample matrix according to the iterative formula, obtains the optimal characteristic matrix, and obtains the optimal characteristic vector. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0029] Figure 1 This is a workflow diagram of the VR metaverse gesture recognition virtual interaction algorithm based on Leapmotion of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0031] Figure 1 As shown, according to one aspect of the present invention, a VR metaverse gesture recognition virtual interaction algorithm based on Leapmotion is provided, comprising the following steps:

[0032] Step 1: Data collection: Use the Leap Motion gesture controller to collect dynamic information of the user's hand, such as finger speed, acceleration, angle, etc.

[0033] Step 2: Feature extraction: pre-process and extract features of the collected hand dynamic information to extract representative features, such as the degree of finger bending, the position and direction of the palm, etc.

[0034] Step 3: Gesture recognition: Input the extracted features into the gesture recognition algorithm and use the trained classification model to determine the type of user gesture. Common gesture recognition algorithms include K-nearest neighbor algorithm, support vector machine, random forest and deep learning model.

[0035] Step 4: Virtual interaction: trigger corresponding virtual interaction operations based on the classification results of the gesture; for example, users can use gesture controllers to move, rotate, and scale objects in the VR metaverse, or perform gesture interaction operations such as clicking, dragging, and zooming in.

[0036] This enables virtual reality interaction, allowing users to perform interactive operations through gestures, such as grabbing, placing, dragging, etc. This interaction method can provide a more intuitive and natural experience and enhance the user's sense of immersion.

[0037] Furthermore, the user gesture in step one can be captured by tracking the position of the hand. The position formula can be expressed as: p = (x, y, z), where p represents the position of the hand and (x, y, z) represents the coordinates in three-dimensional space. In this way, the spatial coordinates of the hand can be calculated to capture the gesture.

[0038] Furthermore, the speed of the user's finger in step one refers to the distance the hand moves in unit time, and the formula can be expressed as: v = (p2-p1) / t, where v represents the speed, p1 and p2 represent the hand positions at two time points, and t represents the time interval. In this way, the distance the finger moves in a period of time can be calculated, so that gesture capture is more accurate.

[0039] Furthermore, the acceleration of the user's finger in step one refers to the rate of change of the hand's speed per unit time, and its formula can be expressed as: a=(v2-v1) / t, where a represents acceleration, v1 and v2 represent speeds at two time points, and t represents the time interval. In this way, the speed change of the finger can be calculated to better capture gestures.

[0040] Furthermore, the direction of the user's palm in step 2 refers to the orientation of the hand in three-dimensional space, and its formula can be expressed as: d = (dx, dy, dz), where d represents the direction vector, and (dx, dy, dz) represents the direction component in three-dimensional space, so that the direction of the palm can be captured.

[0041] Furthermore, the steps of feature extraction in step 3 are:

[0042] S1. Establish a sample matrix based on the user hand data collected in step 1;

[0043] S2. Use the sample matrix as the collected data of the non-smooth non-negative matrix decomposition algorithm to obtain the objective function of the non-smooth non-negative matrix decomposition algorithm

[0044]

[0045]

[0046] X∈R m×n

[0047] Where V is the normalized sample matrix, V j Corresponding to the jth V value, X is the sample vector, X i Corresponding to the i-th X value, H is the feature representation matrix, W is the feature basis matrix, ‖‖ F represents the f-norm of the matrix, θ is a hyperparameter for adjusting the sparsity of the solution, θ∈[0,1], I represents the identity matrix, r is the dimension of the eigenvector, ‖ represents a vector of all 1s, ‖ T represents the transpose of the vector ‖, R represents the sample vector matrix, m and n represent the rows and columns in the sample vector respectively;

[0048] S3, construct the proximal function according to the objective function, and

[0049]

[0050] Find the optimal sample matrix, (where L is the Lipschitz constant, L = ||STWTWS||, <> represents the inner product of the matrix);

[0051] S4. Iterative formula for constructing sample matrix based on optimal sample matrix

[0052]

[0053] According to the iterative formula, the sample matrix is ​​iteratively updated to obtain the feature matrix, where β 0 is a constant, giving a random initial value of iteration β 0 =1,Y 0 =H 0 , iterate according to the iterative formula, and when the number of iterations reaches the set threshold k, the iteration is terminated and H is obtained. k That is the characteristic matrix, k represents the product of the matrix, Y k is an intermediate parameter, P() represents the projection gradient algorithm, and P(Z) means projecting all negative numbers in the matrix Z to 0.

[0054] In this way, the optimal sample matrix is ​​obtained according to the proximal function, the iterative formula of the sample matrix is ​​constructed according to the optimal sample matrix, the sample matrix is ​​iteratively updated according to the iterative formula, the optimal feature matrix is ​​obtained, and finally the optimal feature vector is obtained, thereby improving the accuracy of gesture recognition.

[0055] The electrical components that appear in this article are all electrical components that exist in reality.

[0056] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above-mentioned changes, modifications, additions or substitutions made by ordinary technicians in the technical field within the essential scope of the present invention, which also belong to the protection scope of the present invention.

Claims

1. VR Metaverse Gesture Recognition Virtual Interaction Algorithm Based on Leapmotion, It is characterized in that The following steps are involved: Step 1: Data collection: Use the Leap Motion gesture controller to collect dynamic information of the user's hand; Step 2: Feature extraction: preprocessing and feature extraction of the collected hand dynamic information; Step 3: Gesture recognition: Input the extracted features into the gesture recognition algorithm and use the trained classification model to determine the type of user gesture; Step 4: Virtual interaction: trigger corresponding virtual interaction operations based on the classification results of the gesture.

2. The VR Metaverse Gesture Recognition Virtual Interaction Algorithm based on Leapmotion according to claim 1, Features: The user gesture in the step one can capture the gesture by tracking the position of the hand, and the position formula can be expressed as: p = (x, y, z), where p represents the position of the hand and (x, y, z) represents the coordinates in three-dimensional space.

3. The VR Metaverse gesture recognition virtual interaction algorithm based on Leapmotion according to claim 2, Features: The speed of the user's finger in step one refers to the distance the hand moves in unit time, and its formula can be expressed as: v = (p2-p1) / t, wherein v represents the speed, p1 and p2 represent the hand positions at two time points, and t represents the time interval.

4. The VR metaverse gesture recognition virtual interaction algorithm based on Leapmotion according to claim 3, Features: The acceleration of the user's finger in step 1 refers to the rate of change of the hand's speed per unit time, and its formula can be expressed as: a=(v2-v1) / t, wherein a represents acceleration, v1 and v2 represent speeds at two time points, and t represents the time interval.

5. The VR metaverse gesture recognition virtual interaction algorithm based on Leapmotion according to claim 4, Features: The direction of the user's palm in step 2 refers to the orientation of the hand in three-dimensional space, and its formula can be expressed as: d = (dx, dy, dz), where d represents the direction vector and (dx, dy, dz) represents the direction component in three-dimensional space.

6. The VR metaverse gesture recognition virtual interaction algorithm based on Leapmotion according to claim 5, Features: The steps of feature extraction in step 3 are: S1. Establishing a sample matrix according to the user hand data collected in step 1; S2, using the sample matrix as the collected data of the non-smooth non-negative matrix decomposition algorithm, obtaining the objective function of the non-smooth non-negative matrix decomposition algorithm X∈R m×n Where V is the normalized sample matrix, V j Corresponding to the jth V value, X is the sample vector, X i Corresponding to the i-th X value, H is the feature representation matrix, W is the feature basis matrix, || || F represents the f-norm of the matrix, θ is a hyperparameter that adjusts the sparsity of the solution, θ∈[0,1], I represents the identity matrix, r is the dimension of the eigenvector, || represents a vector of all 1s, || T represents the transpose of the vector ||, R represents the sample vector matrix, m and n represent the rows and columns in the sample vector respectively; S3, constructing a proximal function according to the objective function, and constructing a proximal function according to the proximal function Find the optimal sample matrix, (where L is the Lipschitz constant, L = ||STWTWS||, <> represents the inner product of the matrix); S4. Iterative formula for constructing a sample matrix based on the optimal sample matrix The sample matrix is ​​iteratively updated according to the iterative formula to obtain a feature matrix, where β 0 is a constant, giving a random initial value of iteration β 0 =1,Y 0 =H 0 , iteratively update according to the iterative formula, when the number of iterations reaches the set threshold k, the iteration is terminated, and H is obtained k That is, the characteristic matrix, k represents the matrix product, Y is the intermediate parameter, and Y k Corresponding to the kth Y value, P() represents the projected gradient algorithm, and P(Z) represents projecting all negative numbers in the matrix Z to 0.

Images