Filtering method, device and system for improving kneading action stability in VR gesture tracking and application

Abstract

The invention relates to a filtering method, device and system for improving kneading action stability in VR gesture tracking and application. A kneading action state classification model is established; real-time three-dimensional coordinates of a thumb fingertip joint point and an index finger fingertip joint point are obtained, the distance between the thumb fingertip joint point and the index finger fingertip joint point is calculated, and kneading action state classification is conducted through the model; on the basis of different kneading action state categories, adaptive filtering processing is executed, and the data of the thumb fingertip joint point and the index finger fingertip joint point after filtering processing is used for driving a virtual hand model; the filtering device and the filtering system are realized based on the method, and are applied to gesture tracking in VR human-computer interaction. According to the method, different stages of the kneading action are accurately identified, and an accurate context basis is provided for switching of filtering strategies; dynamically associating the identified kneading state with the filtering intensity to realize intelligent switching; and unification of calculation efficiency and problem pertinence is realized.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing, and in particular to a filtering method, apparatus, system, and application for improving the stability of pinching motions in VR gesture tracking. Background Technology

[0002] Virtual reality technology aims to provide users with an immersive interactive experience, and natural, intuitive gesture interaction is key to achieving a high degree of immersion. In recent years, gesture tracking technology based on computer vision has made significant progress. This technology captures images of the user's hand using sensors such as depth cameras and uses trained machine learning models (such as convolutional neural networks) to infer the three-dimensional spatial coordinates of multiple joints of the hand in real time, thereby driving the virtual hand model to complete various movements.

[0003] However, in practical applications, the quality of raw joint data acquired from tracking systems is constrained by multiple factors—sensors themselves have inherent electrical noise; changes in ambient lighting and occlusion can affect image quality and model recognition accuracy; machine learning models inherently have certain errors in estimation under complex postures; and the unavoidable physiological tremors of the human hand during fine manipulation contribute to the high-frequency jitter and random noise in the raw coordinate data. While this jitter is tolerable in large-scale gestures such as grasping and dragging, it is drastically amplified in the precise pinching and releasing actions of the thumb and forefinger, where accuracy and stability are extremely important. Directly using jitter data to drive the virtual hand will cause noticeable "flickering," "jumping," or "sticking" phenomena near the pinching threshold, severely damaging the user's experience of accurately grasping, manipulating, and releasing virtual objects, and reducing the credibility and efficiency of the interaction.

[0004] To address the jitter issue, existing technologies generally employ fixed, non-adaptive post-processing filtering algorithms. A common approach is to uniformly apply low-pass filters, such as moving average filtering, exponential smoothing filtering, or Kalman filtering, to the data streams of all relevant nodes. While this method can smooth out some noise, its static parameter strategy fails to differentiate between different user interaction intentions and action stages. When the user intends to move quickly or make large gestures, strong filtering introduces perceptible delays, causing virtual hand "ghosting" or sluggish responses, disrupting the responsiveness of the interaction. Conversely, when the user intends to perform fine pinch gestures, weak filtering is insufficient to suppress jitter and cannot provide a stable pinch posture. Therefore, fixed filtering parameters are always caught in a dilemma between "delay" and "stability," unable to be dynamically optimized based on the actual interaction context. Summary of the Invention

[0005] This invention addresses the problems existing in the prior art and provides a filtering method, device, system, and application for improving the stability of pinching actions in VR gesture tracking. For the pinching action of the thumb and index finger, an adaptive filtering strategy is used to effectively suppress jitter and improve the stability and reliability of the action.

[0006] The technical solution adopted in this invention is a filtering method to improve the stability of pinching motion in VR gesture tracking, and to establish a pinching motion state classification model. The real-time three-dimensional coordinates of the thumb and index finger joints are obtained and the distance between them is calculated. The pinching action state is then classified using the model. Based on different pinching action state categories, adaptive filtering is performed, and the filtered data of the thumb and index finger joints are used to drive the virtual hand model.

[0007] Preferably, the kneading action state classification model is a dual-threshold classification model, with dual thresholds T... loose and T tight The kneading action state is classified into three-dimensional spatial states based on real-time three-dimensional spatial distance: free movement state, pre-kneading state, and tight kneading state; 0 < T tight <T loose .

[0008] Preferably, the filtering intensity increases sequentially for the free-moving state, the pre-kneading state, and the tightly kneading state.

[0009] Preferably, in the free-movement state, exponential smoothing filtering with a smoothing coefficient α1 or filtering with the smoothing factor turned off is used; In the pre-kneading state, an exponential smoothing filter with a smoothing coefficient α2 or a moving average filter with a window size of W1 is used. In the tightly kneaded state, an exponential smoothing filter with a smoothing coefficient of α3, a moving average filter with a window size of W2, or a Kalman filter is enabled. α1 < α2 < α3; W1 < W2.

[0010] Preferably, a hysteresis mechanism is introduced during state transition.

[0011] Preferably, the hysteresis mechanism is to set the trigger threshold for state switching to be higher than the threshold for state entry.

[0012] Preferably, the threshold for switching from the pre-kneading state to the free-moving state is T. loose +Δh1, Δh1>0; The exit threshold for switching from the squeezed state to the pre-squeezed state is set to T. tight +Δh2, Δh2> 0.

[0013] A filtering device for improving the stability of pinching motions in VR gesture tracking, comprising: The data acquisition and calculation module is used to collect the real-time three-dimensional positions of the thumb and index finger joints and calculate the spatial distance between them. A kneading action state classification module is used to classify the kneading action state using the kneading action state classification model. The adaptive filtering module is used to perform adaptive filtering based on different kneading action state categories; The data output module is used to output the filtered data of the thumb and index finger joints to drive the virtual hand model.

[0014] A filtering system for improving the stability of pinching motions in VR gesture tracking includes a processor and a memory. The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the filtering method for improving the stability of pinching motions in VR gesture tracking according to the instructions in the program code.

[0015] An application of the filtering method described above for improving the stability of pinching motions in VR gesture tracking, applied to gesture tracking in VR human-computer interaction.

[0016] This invention relates to a filtering method, device, system, and application for improving the stability of pinching actions in VR gesture tracking. The method establishes a pinching action state classification model; acquires the real-time three-dimensional coordinates of the thumb and index finger joints and calculates the distance between them, using the model to classify pinching action states; performs adaptive filtering based on different pinching action state categories, and uses the filtered data of the thumb and index finger joints to drive a virtual hand model; implements a filtering device and system based on the method, and applies it to gesture tracking in VR human-computer interaction.

[0017] The beneficial effects of this invention are as follows: (1) By setting multiple distance thresholds, different stages of the kneading action can be accurately identified, providing a precise contextual basis for switching filtering strategies; (2) The identified kneading state is dynamically associated with the filtering intensity. When moving freely, weak filtering is used to ensure the response speed; when kneading pre-, medium filtering is used to balance the performance; and when kneading tightly, strong filtering is used to maximize stability, thus realizing intelligent switching. (3) Focus adaptive filtering resources on the thumb and index finger joints that are most critical to the pinching experience, rather than the whole hand data, to achieve a balance between computational efficiency and problem targeting. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the filtering device structure of the present invention. Detailed Implementation

[0019] The present invention will be further described in detail below with reference to embodiments, but the scope of protection of the present invention is not limited thereto.

[0020] This invention relates to a filtering method for improving the stability of pinching actions in VR gesture tracking. By recognizing the pinching state of the thumb and index finger in real time, the filtering parameters are dynamically switched or adjusted, thereby ensuring low latency in gesture tracking in the non-pinching state and significantly improving the stability of the fingertip position in the pinching preparation and maintenance state, ultimately achieving a precise, stable, and smooth pinching interaction experience.

[0021] The method of the present invention includes the following steps: (1) Establish a classification model for the state of kneading action; (2) Obtain the real-time three-dimensional coordinates of the thumb tip joint and the index finger tip joint and calculate the distance between them, and classify the pinching action state using the model. (3) Based on different pinching action state categories, perform adaptive filtering processing, and use the filtered data of the thumb fingertip joint and the index finger fingertip joint to drive the virtual hand model.

[0022] First, this invention establishes a classification model for the state of kneading actions.

[0023] Specifically, the kneading action state classification model is a dual-threshold classification model, using a dual-threshold T... loose and T tight The kneading action state is classified into three-dimensional spatial states based on real-time three-dimensional spatial distance: free movement state, pre-kneading state, and tight kneading state; 0 < T tight <T loose ; That is, when the real-time three-dimensional spatial distance (between fingers) is greater than T loose When the hand is in a free-moving state, the interaction intent is mainly based on fast, wide-range movements, prioritizing low latency; when the real-time three-dimensional spatial distance (between fingers) is less than or equal to T... loose And greater than T tight When the fingers are in a pre-pinch state, it is determined that the user is preparing to perform a fine pinch, and a balance between stability and responsiveness needs to be struck; while when the real-time three-dimensional spatial distance (between the fingers) is less than or equal to T... tight When the fingers are in a tightly pinched state, the user is performing or maintaining a fine pinch, which requires the highest stability of the fingertip position.

[0024] In practical applications, the threshold can be further subdivided to construct a multi-threshold classification model.

[0025] Subsequently, the system continuously acquires the real-time three-dimensional coordinates of the thumb and index finger joints output by the gesture tracking algorithm, calculates the Euclidean distance between these two points, and classifies the pinching action state using the model, which serves as the core basis for judging the pinching state and filtering intensity.

[0026] Finally, based on different pinching action state categories, adaptive filtering is performed, and the filtered data of the thumb and index finger joints are used to drive the virtual hand model.

[0027] Specifically, the system dynamically selects or adjusts the filtering algorithm parameters for the fingertip joints of the thumb and index finger based on the currently determined pinching state, with the filtering intensity increasing successively for the free movement state, the pre-pinch state, and the tight pinch state. The filtering algorithm includes one or more of exponential smoothing filtering, moving average filtering, Kalman filtering, and complementary filtering, with different filtering algorithms configured for different states, and / or different parameters configured for the same filtering algorithm.

[0028] In the free movement state, a weak filtering or direct penetration mode is adopted. For example, an exponential smoothing filter with a small smoothing coefficient is used, or the original data is used directly to minimize latency and ensure that the gestures are responsive. In this embodiment, an exponential smoothing filter with a smoothing coefficient α1 or a filter is turned off is adopted. In the pre-kneading state, a medium-intensity filter is used, for example, increasing the smoothing coefficient of the exponential smoothing filter, or switching to a moving average filter with a suitable window size, to start smoothing the jitter and prepare for possible fine operations; in this embodiment, an exponential smoothing filter with a smoothing coefficient of α2 or a moving average filter with a window size of W1 is used. In the tightly pinched state, a strong filtering mode is adopted, such as using an exponential smoothing filter with a large smoothing coefficient, or switching to a more complex filtering algorithm such as a Kalman filter with a larger window, to strongly suppress jitter and ensure the stable maintenance of the pinching posture; in this embodiment, an exponential smoothing filter with a smoothing coefficient of α3 or a moving average filter with a window size of W2 or a Kalman filter is enabled is adopted. α1 < α2 < α3; W1 < W2; In this embodiment, 0 ≤ α1 < 0.3, 0.3 ≤ α2 < 0.7, 3 ≤ W1 < 7, 0.7 ≤ α3 ≤ 1, and W2 ≥ 7.

[0029] In this embodiment, the relationship between the filter strength coefficient K and the distance d is as follows: when in the pre-kneading state, K = (dT) tight ) / (T loose -Ttight This enables a smooth transition from strong to weak filtering. However, when the overall motion speed is higher than the set value, the filtering intensity of all states should be appropriately reduced to ensure responsiveness.

[0030] The stabilized thumb and index finger joint data, after adaptive filtering, along with other hand joint data, are input into the virtual hand skeleton driving system to generate a smooth, stable, and jitter-free virtual hand model for rendering.

[0031] Furthermore, the present invention introduces a hysteresis mechanism during state switching.

[0032] The hysteresis mechanism involves setting the trigger threshold for state switching to be higher than the threshold for entering a new state.

[0033] The threshold for switching from the pre-kneading state to the free-moving state is T. loose +Δh1, Δh1>0; The exit threshold for switching from the squeezed state to the pre-squeezed state is set to T. tight +Δh2, Δh2> 0.

[0034] By setting the trigger threshold for state switching slightly higher than the threshold for state entry, this invention can avoid frequent state jumps caused by minor data fluctuations near the threshold, thereby ensuring the continuity and stability of the filtering strategy.

[0035] In fact, the present invention can further optimize the method, such as predicting or correcting the state category determination result based on the historical trend of the Euclidean distance. When the distance decreases rapidly and continuously in a short period of time, even if T is not reached... tight Filtering strategies that are either adopted in advance or gradually enhanced to a tight-knead state are also used.

[0036] The present invention also relates to a filtering device for improving the stability of pinching motions in VR gesture tracking, comprising: The data acquisition and calculation module is used to collect the real-time three-dimensional positions of the thumb and index finger joints and calculate the spatial distance between them. A kneading action state classification module is used to classify the kneading action state using the kneading action state classification model. The adaptive filtering module is used to perform adaptive filtering based on different kneading action state categories; The data output module is used to output the filtered data of the thumb and index finger joints to drive the virtual hand model.

[0037] This invention also relates to a filtering system for improving the stability of pinching motions in VR gesture tracking, comprising a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the filtering method for improving the stability of pinching motions in VR gesture tracking according to the instructions in the program code.

[0038] This invention also relates to the application of the filtering method described above for improving the stability of pinching motions in VR gesture tracking, which is applied to gesture tracking in VR human-computer interaction.

[0039] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0040] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0041] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0042] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0043] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0044] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A filtering method for improving the stability of pinching motions in VR gesture tracking, characterized in that: Establish a classification model for the state of kneading action; The real-time three-dimensional coordinates of the thumb and index finger joints are obtained and the distance between them is calculated. The pinching action state is then classified using the model. Based on different pinching action state categories, adaptive filtering is performed, and the filtered data of the thumb and index finger joints are used to drive the virtual hand model.

2. The filtering method for improving the stability of pinching motions in VR gesture tracking according to claim 1, characterized in that: The kneading action state classification model is a dual-threshold classification model, with dual thresholds T loose and T tight The kneading action state is classified into three-dimensional spatial states based on real-time three-dimensional spatial distance: free movement state, pre-kneading state, and tight kneading state; 0 < T tight <T loose .

3. The filtering method for improving the stability of pinching motions in VR gesture tracking according to claim 2, characterized in that: The filtering intensity increases sequentially for the free-moving state, the pre-kneading state, and the tightly kneading state.

4. The filtering method for improving the stability of pinching motions in VR gesture tracking according to claim 3, characterized in that: In the free-movement state, exponential smoothing filtering with a smoothing coefficient α1 or filtering with the filter off is used. In the pre-kneading state, an exponential smoothing filter with a smoothing coefficient α2 or a moving average filter with a window size of W1 is used. In the tightly kneaded state, an exponential smoothing filter with a smoothing coefficient of α3, a moving average filter with a window size of W2, or a Kalman filter is enabled. α1 < α2 < α3; W1 < W2.

5. The filtering method for improving the stability of pinching motions in VR gesture tracking according to claim 1, characterized in that: Introduce a hysteresis mechanism during state transitions.

6. The filtering method for improving the stability of pinching motions in VR gesture tracking according to claim 5, characterized in that: The hysteresis mechanism involves setting the trigger threshold for state switching to be higher than the threshold for entering a new state.

7. A filtering method for improving the stability of pinching motions in VR gesture tracking according to claim 6, characterized in that: The threshold for switching from the pre-kneading state to the free-moving state is T. loose +Δh1, Δh1>0; The exit threshold for switching from the squeezed state to the pre-squeezed state is set to T. tight +Δh2, Δh2> 0.

8. A filtering device for improving the stability of pinching motions in VR gesture tracking, characterized in that: include: The data acquisition and calculation module is used to collect the real-time three-dimensional positions of the thumb and index finger joints and calculate the spatial distance between them. A kneading action state classification module is used to classify the kneading action state using the kneading action state classification model. The adaptive filtering module is used to perform adaptive filtering based on different kneading action state categories; The data output module is used to output the filtered data of the thumb and index finger joints to drive the virtual hand model.

9. A filtering system for improving the stability of pinching motions in VR gesture tracking, characterized in that: Including the processor and memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the filtering method for improving the stability of pinching motion in VR gesture tracking as described in any one of claims 1 to 7, according to the instructions in the program code.

10. An application of the filtering method for improving the stability of pinching motions in VR gesture tracking as described in any one of claims 1 to 7, characterized in that: Gesture tracking applied in VR human-computer interaction.

Images