A high-precision handle interaction method based on adaptive filtering and deep learning

Abstract

The application discloses a high-precision handle interaction method based on adaptive filtering and deep learning, and aims to solve the problems of jitter, delay and insufficient target selection accuracy of the existing handle interaction. The method comprises the following steps: first, the original position input of the handle is unified through coordinate transformation; then, the One Euro filtering is adopted, the cut-off frequency is dynamically adjusted based on the handle speed, low-speed anti-jitter and high-speed hand-following are realized; the LSTM model is used to learn the historical position and speed characteristics, the future trajectory is predicted to compensate for the system delay; the current smooth position and the predicted position are dynamically fused according to the current speed; the user's intention is determined through the distance between the continuous frames, and the target is accurately selected by combining the magnetic attraction locking. The method does not require additional hardware, and is suitable for VR / AR, virtual sand table and other scenes, and can improve the interaction stability and accuracy.

Description

Technical Field

[0001] This invention relates to the fields of virtual reality (VR), augmented reality (AR), and human-computer interaction technology, specifically to a high-precision wireless controller interaction method based on adaptive filtering and deep learning, which is applicable to scenarios in virtual scenes that require high-precision pointing and stable interaction, such as target selection, model editing, spatial drawing, and menu operation. Background Technology

[0002] In virtual interactive systems such as VR / AR and holographic sandboxes, wireless controllers are the core devices for users to interact with virtual scenes. They emit virtual rays to perform operations such as pointing, grabbing, dragging, and fine-grained model editing. However, existing wireless controller interaction technology faces three major pain points that seriously affect the interactive experience:

[0003] 1. Signal jitter and noise interference: Factors such as unstable wireless signal transmission, physiological hand tremors, and noise of the inertial measurement unit (IMU) itself cause significant fluctuations in the original position data of the handle. When stationary or moving slowly, these fluctuations are amplified, causing cursor drift and unstable pointing, making it difficult to accurately aim at small targets.

[0004] 2. System latency and responsiveness contradiction: There is an inherent physical latency from sensor data acquisition and attitude calculation to screen rendering. Traditional interaction methods directly use the current frame data to construct virtual rays, which causes the virtual ray position to lag behind the user's actual hand movements when the controller is quickly swung, resulting in spatiotemporal misalignment and misjudgment.

[0005] 3. Limitations of existing solutions: To improve accuracy, some solutions use external optical positioning technology (such as HTC Vive Lighthouse), but these rely on dedicated hardware, resulting in high deployment costs and susceptibility to occlusion, which can lead to positional shifts. Traditional filtering methods (such as fixed-parameter low-pass filtering and Kalman filtering) cannot balance stability and real-time performance, with insufficient smoothing at low speeds or significant time lag at high speeds. Existing research based on deep learning focuses on gesture recognition or pose estimation, lacking time-series prediction and dynamic compensation mechanisms for the low-latency requirements of controller interaction, making direct application difficult.

[0006] Therefore, there is an urgent need for a high-precision controller interaction method that requires no additional hardware, can dynamically balance anti-shake and responsiveness, and effectively compensate for system latency, in order to solve the problems of significant shaking, obvious latency, and insufficient target selection accuracy in existing technologies. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing wireless gamepad interaction technology and provide a high-precision gamepad interaction method based on adaptive filtering and deep learning. Through the end-to-end design of "coordinate unification - dynamic anti-shake - latency compensation - position optimization - precise locking", a low-shake, low-latency, and high-precision interactive experience is achieved.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following solution:

[0009] A high-precision handle interaction method based on adaptive filtering and deep learning includes the following steps:

[0010] S1. Coordinate Transformation and Input Preprocessing: Obtain the original three-dimensional observation position of the wireless handle in the tracking coordinate system in the discrete time domain, and map the original three-dimensional observation position to the target model coordinate system through a pre-calibrated rigid body transformation matrix;

[0011] S2. Adaptive filtering: The instantaneous speed of the handle is estimated based on adjacent sampling points. The instantaneous speed is smoothed by low-pass filtering to obtain the smoothed speed. The cutoff frequency of the position filter is dynamically adjusted according to the magnitude of the smoothed speed, and then the current smoothed position of the handle is updated.

[0012] S3, LSTM trajectory prediction: Construct a feature sequence containing smoothed positions and smoothed velocities from multiple historical frames, input the feature sequence into a pre-trained LSTM neural network model, and output a predicted position sequence for future frames;

[0013] S4. Position fusion: Dynamically adjust the prediction weights according to the current handle speed, and linearly fuse the current smooth position with the predicted position of the corresponding delay time step to obtain the fused position;

[0014] S5. Ray Construction and Target Locking: Starting from the fusion position, a virtual ray is constructed using the unit vector pointing in the direction of the handle. If the distance between the target object and the virtual ray continuously meets the set threshold, the magnetic locking logic is triggered, and the cursor is smoothly attracted to the center of the target object, achieving high-precision target selection.

[0015] Compared to existing controller interaction processes, this invention represents a substantial improvement in overall processing flow, data processing methods, and latency compensation mechanisms. Traditional processes typically employ only a fixed-parameter low-pass filter to smooth the controller position once, then directly calculate the ray direction and render based on the single-frame result. This process involves fewer steps, lacks adaptive adjustment capabilities for motion states, and does not include any prediction mechanisms or directional stability constraints. Therefore, significant jitter is likely to occur in low-speed scenes, while noticeable lag occurs during high-speed motion. Furthermore, in the presence of system rendering latency or network latency, the controller pointing will deviate and drift, and the ray direction is highly susceptible to transient noise interference. This is particularly problematic when operating on small targets or in densely populated areas, where target selection errors are significant, resulting in an unstable overall interactive experience.

[0016] In contrast, this invention adds and integrates several key steps, such as adaptive One Euro filtering and temporal trajectory prediction, to the original workflow, forming a continuous multi-level real-time processing pipeline. In the filtering stage, the adaptive dynamic filtering strategy employed in this invention automatically adjusts the cutoff frequency according to the controller speed. This significantly suppresses noise and minor jitter at low speeds while maintaining sufficient tracking accuracy at high speeds, effectively solving the problem of the old workflow having to trade off between stability and real-time performance. Regarding latency compensation, this invention introduces short-term position prediction based on historical trajectory trends, adding feedforward compensation when calculating the ray origin. This allows the system to maintain a trajectory performance close to the real controller movement even with rendering or network latency, significantly reducing offset and lag. Furthermore, this invention stabilizes the ray direction based on the predicted position, effectively suppressing the directional jumps and micro-jitter common in the old workflow, enabling the controller to maintain higher accuracy and stability when pointing at small or densely packed target areas.

[0017] Overall, this invention not only adds key steps missing from the old process, but also adopts entirely new technical mechanisms in several core stages such as filtering and prediction. This upgrades the data processing pipeline from a single fixed filtering and direct ray casting to a multi-level adaptive high-precision pipeline combining dynamic filtering and time-series prediction. Based on this, the system significantly outperforms existing technologies in trajectory stability, noise resistance, latency compensation, and target pointing accuracy, achieving a high-precision, low-latency, and high-stability interactive effect that is difficult to achieve simultaneously in traditional processes.

[0018] Preferably, the coordinate transformation and input preprocessing in step S1 specifically include:

[0019] The original handle point at each moment is expanded into a homogeneous vector. The rigid body transformation matrix includes a 3×3 rotation matrix and a 1×3 translation vector. The homogeneous vector is transformed by the rigid body transformation matrix to obtain the unified coordinate position in the target model coordinate system.

[0020] Preferably, the adaptive filtering process in step S2 specifically includes:

[0021] The adaptive filtering is a One Euro filter;

[0022] The instantaneous velocity is calculated based on the position difference between adjacent sampling points and the sampling period. The instantaneous velocity is then smoothed using a fixed cutoff frequency to obtain a smoothed velocity.

[0023] Based on the preset minimum cutoff frequency, speed coefficient and the modulus of the smoothing speed, an adaptive cutoff frequency is calculated. At low speeds, the cutoff frequency is reduced to enhance the smoothing effect, and at high speeds, the cutoff frequency is increased to reduce delay.

[0024] The position smoothing factor is calculated based on the adaptive cutoff frequency and the sampling frequency, and the current smoothed position is obtained through iterative update.

[0025] Preferably, the LSTM trajectory prediction in step S3 specifically includes:

[0026] The feature sequence consists of continuous frame features of a length equal to a preset historical window, and each frame feature is a combination of smooth position and smooth speed at the corresponding moment.

[0027] After being pre-trained, the LSTM neural network model receives the feature sequence and outputs a predicted position sequence for a preset number of future frames. The predicted position sequence reflects the short-term movement trend of the handle.

[0028] Preferably, the position fusion prediction weight adjustment logic in step S4 is as follows: when the handle speed is ≥ 0.4m / s, the prediction weight value tends to 1, and the system adopts the predicted position; when the handle speed is < 0.03m / s, the prediction weight value tends to 0, and the system adopts the current smooth position; when the handle speed is ≥ 0.03m / s and < 0.4m / s, the prediction weight changes continuously and linearly with the handle speed.

[0029] Preferably, the ray construction in step S5 specifically includes:

[0030] The virtual ray starts at the fusion position and takes the unit pointing vector obtained from the handle posture calculation as its direction, and is represented in parametric form. The point of action of the ray on the interaction plane is obtained by solving the intersection equation of the virtual ray and the interaction plane.

[0031] Preferably, the target locking in step S5 specifically includes:

[0032] If the minimum distance between the target object and the virtual ray is less than a set threshold within a consecutive preset number of frames, it is determined that the user intends to point at the target object; the cursor is smoothly attracted from the point of action of the ray to the center of the target object by controlling the preset adsorption intensity parameter, so as to achieve stable locking.

[0033] Preferably, the operating parameters in the discrete time domain satisfy the following: the sampling frequency is the reciprocal of the sampling period, and the system periodically acquires the original three-dimensional observation position of the handle according to the sampling period.

[0034] The beneficial effects of this invention are as follows:

[0035] 1. Dynamic anti-shake, balancing stability and responsiveness: Adopting an adaptive One Euro filter, the cutoff frequency of the position filter is dynamically adjusted by the smoothing speed. Strong smoothing at low speed / stationary time to suppress noise and hand tremor, and weak smoothing at high speed to reduce delay, which solves the contradiction of traditional fixed parameter filtering: "stable but lagging, fast but jittery".

[0036] 2. Delay compensation to enhance interactive foresight: The LSTM time-series prediction model is introduced to predict the future short-term trajectory based on the feature sequence of historical smooth position and velocity. The predicted position is used as the delay compensation amount through position fusion to effectively offset the physical delay of the system, so that the virtual ray is synchronized with the user's real hand movements and avoids spatiotemporal misalignment during rapid operation.

[0037] 3. Precise targeting and optimized interaction for small targets: The design of the "continuous frame distance judgment + smooth magnetic locking" mechanism eliminates instantaneous noise interference through multi-frame verification, and then locks the cursor to the center of the target through smooth magnetic locking, which significantly improves the selection hit rate of small targets or dense target areas and reduces the problems of misselection and missed selection.

[0038] 4. Lightweight deployment, no additional hardware required: High-precision interaction is achieved solely through software algorithm optimization, based on the existing sensor data (position, attitude) of ordinary wireless controllers, without the need to add external optical base stations, multimodal sensors, or other hardware, reducing deployment costs and the barrier to entry for use;

[0039] 5. High adaptability and wide scene coverage: Through configurable parameters (such as historical window length N, prediction frame number K, adsorption intensity γ, etc.), it can adapt to systems with different latency characteristics and interactive scenarios with different accuracy requirements (such as virtual button clicks, holographic sand table annotation, fine editing of 3D models, etc.). Attached Figure Description

[0040] Figure 1 This is a flowchart of a high-precision handle interaction method based on adaptive filtering and deep learning.

[0041] Figure 2A comparison chart of test metrics for different interaction methods. Detailed Implementation

[0042] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0043] like Figure 1 As shown, Figure 1 This is a flowchart of a high-precision handle interaction method based on adaptive filtering and deep learning according to the present invention. The method includes the following steps:

[0044] S1. Coordinate Transformation and Input Preprocessing: Obtain the original three-dimensional observation position of the wireless handle in the tracking coordinate system in the discrete time domain, and map the original three-dimensional observation position to the target model coordinate system through a pre-calibrated rigid body transformation matrix;

[0045] S2. Adaptive filtering: The instantaneous speed of the handle is estimated based on adjacent sampling points. The instantaneous speed is smoothed by low-pass filtering to obtain the smoothed speed. The cutoff frequency of the position filter is dynamically adjusted according to the magnitude of the smoothed speed, and then the current smoothed position of the handle is updated.

[0046] S3, LSTM trajectory prediction: Construct a feature sequence containing smoothed positions and smoothed velocities from multiple historical frames, input the feature sequence into a pre-trained LSTM neural network model, and output a predicted position sequence for future frames;

[0047] S4. Position fusion: Dynamically adjust the prediction weights according to the current handle speed, and linearly fuse the current smooth position with the predicted position of the corresponding delay time step to obtain the fused position;

[0048] S5. Ray Construction and Target Locking: Starting from the fusion position, a virtual ray is constructed using the unit vector pointing in the direction of the handle. If the distance between the target object and the virtual ray continuously meets the set threshold, the magnetic locking logic is triggered, and the cursor is smoothly attracted to the center of the target object, achieving high-precision target selection.

[0049] Preferably, the coordinate transformation and input preprocessing in step S1 specifically include:

[0050] The original handle point at each moment is expanded into a homogeneous vector. The rigid body transformation matrix includes a 3×3 rotation matrix and a 1×3 translation vector. The homogeneous vector is transformed by the rigid body transformation matrix to obtain the unified coordinate position in the target model coordinate system.

[0051] Preferably, the adaptive filtering process in step S2 specifically includes:

[0052] The adaptive filtering is a One Euro filter;

[0053] The instantaneous velocity is calculated based on the position difference between adjacent sampling points and the sampling period. The instantaneous velocity is then smoothed using a fixed cutoff frequency to obtain a smoothed velocity.

[0054] Based on the preset minimum cutoff frequency, speed coefficient and the modulus of the smoothing speed, an adaptive cutoff frequency is calculated. At low speeds, the cutoff frequency is reduced to enhance the smoothing effect, and at high speeds, the cutoff frequency is increased to reduce delay.

[0055] The position smoothing factor is calculated based on the adaptive cutoff frequency and the sampling frequency, and the current smoothed position is obtained through iterative update.

[0056] Preferably, the LSTM trajectory prediction in step S3 specifically includes:

[0057] The feature sequence consists of continuous frame features of a length equal to a preset historical window, and each frame feature is a combination of smooth position and smooth speed at the corresponding moment.

[0058] After being pre-trained, the LSTM neural network model receives the feature sequence and outputs a predicted position sequence for a preset number of future frames. The predicted position sequence reflects the short-term movement trend of the handle.

[0059] Preferably, the position fusion prediction weight adjustment logic in step S4 is as follows: when the handle speed is ≥ 0.4m / s, the prediction weight value tends to 1, and the system adopts the predicted position; when the handle speed is < 0.03m / s, the prediction weight value tends to 0, and the system adopts the current smooth position; when the handle speed is ≥ 0.03m / s and < 0.4m / s, the prediction weight changes continuously and linearly with the handle speed.

[0060] Preferably, the ray construction in step S5 specifically includes:

[0061] The virtual ray starts at the fusion position and takes the unit pointing vector obtained from the handle posture calculation as its direction, and is represented in parametric form. The point of action of the ray on the interaction plane is obtained by solving the intersection equation of the virtual ray and the interaction plane.

[0062] Preferably, the target locking in step S5 specifically includes:

[0063] If the minimum distance between the target object and the virtual ray is less than a set threshold within a consecutive preset number of frames, it is determined that the user intends to point at the target object; the cursor is smoothly attracted from the point of action of the ray to the center of the target object by controlling the preset adsorption intensity parameter, so as to achieve stable locking.

[0064] Preferably, the operating parameters in the discrete time domain satisfy the following: the sampling frequency is the reciprocal of the sampling period, and the system periodically acquires the original three-dimensional observation position of the handle according to the sampling period.

[0065] This invention proposes a high-precision handle interaction method based on adaptive filtering and deep learning, the specific process of which is as follows:

[0066] 1. System Scenarios and Symbol Conventions

[0067] 1.1 Time and Sampling

[0068] In this invention, the handle interaction system will operate in the discrete time domain.

[0069] Sampling period: Δt (seconds);

[0070] Sampling frequency: f s =1 / Δt (Hertz).

[0071] 1.2 Coordinate System Definition

[0072] Original observation position: The position of the wireless handle in the tracking system coordinate system (W), denoted as:

[0073]

[0074] Target model coordinate system: The target coordinate system for system interaction, denoted as S.

[0075] 2. Technical Description

[0076] This invention provides a high-precision interaction method for wireless controllers based on adaptive filtering and deep learning. The method first obtains the original 3D observation position of the wireless controller in the tracking coordinate system in the discrete time domain, and then maps it to the target model coordinate system through a pre-calibrated rigid body transformation matrix, thus achieving spatial unification of the input.

[0077] Subsequently, the system enters the adaptive One Euro filtering stage to suppress visual jitter. The system estimates the instantaneous speed of the handle based on adjacent sampling points and uses a fixed cutoff frequency to perform low-pass smoothing on the speed. Then, it dynamically calculates the adaptive cutoff frequency of the position filter based on the magnitude of the smoothed speed. That is, it lowers the cutoff frequency at low speeds to enhance the smoothing effect and raises the cutoff frequency at high speeds to reduce delay. Based on this, the current smoothed position of the handle is updated.

[0078] To further compensate for the physical delay of the system, the method introduces an LSTM trajectory prediction mechanism: a feature sequence containing the smoothed position and smoothed velocity of the historical N frames is constructed, input into a pre-trained LSTM neural network model, and the predicted position sequence of the future K frames is output.

[0079] Based on this, dynamic fusion and interactive judgment are performed: the system dynamically adjusts the prediction weight according to the current controller speed, linearly fuses the current smooth position with the predicted position of the corresponding delay time step, and obtains the fused position as the ray starting point; finally, the predicted ray and scene objects are used to perform cone detection. If the distance between a target and the ray meets the set threshold within M consecutive frames, the spatiotemporal locking logic is triggered, and the cursor is smoothly snapped to the center of the target, thereby achieving high-precision selection of the target.

[0080] 3. Technical Process

[0081] 3.1 Coordinate Transformation and Input Preprocessing

[0082] 3.1.1 Homogeneous Coordinates and Rigid Body Transformation

[0083] Assume the rigid body transformation matrix from the tracking coordinate system W to the target model coordinate system S is:

[0084]

[0085] Where R is a 3×3 rotation matrix describing the directional relationship of the coordinate system; t is a 1×3 translation vector representing the translation of the origin of the coordinate system. The "1" in this matrix is ​​a constant term in the homogeneous coordinate transformation, used to ensure the mathematical and geometric integrity and consistency of the rigid body transformation matrix.

[0086] 3.1.2 Coordinate Mapping

[0087] Expand the original handle point at each time step into a homogeneous vector. ):

[0088]

[0089] This formula is a mathematical method for representing an 1 / n-dimensional vector as an 1 / n+1-dimensional vector. By introducing an extra component, it unifies the representation of points and vectors, thus facilitating affine transformations such as translation, scaling, and rotation using matrix operations. When converting from ordinary coordinates to homogeneous coordinates, if the original position (x, y, z) is a point, it becomes (x, y, z, 1).

[0090] The process of mapping to the target model coordinate system is as follows:

[0091]

[0092] Therefore, the input position definition used in subsequent system processing ( )for:

[0093]

[0094] 3.2 Adaptive One Euro Filter

[0095] The One Euro filter is a speed-adaptive first-order low-pass filter. Its core idea is to use strong filtering at low speeds to combat jitter and weak filtering at high speeds to reduce delay.

[0096] 3.2.1 Velocity Estimation

[0097] This invention estimates the current velocity based on adjacent sampling points in discrete time. ):

[0098]

[0099] To avoid noise amplification, the speed itself needs to be low-pass filtered to obtain a smooth speed. ):

[0100]

[0101] in It is determined by a pre-set fixed cutoff frequency. The result is obtained through the following formula:

[0102]

[0103] Where Δt is the sampling period, f s =1 / Δt is the sampling frequency.

[0104] 3.2.2 Adaptive Cutoff Frequency

[0105] Smooth speed modulus (S) t This reflects the speed of the current handle movement and is defined as:

[0106]

[0107] The adaptive cutoff frequency is then:

[0108]

[0109] in:

[0110] f min This is a pre-set fixed minimum cutoff frequency used to control the smoothness at low speeds. The smaller the value, the more obvious the smoothing effect, but the greater the hysteresis delay will also be.

[0111] β is a pre-set fixed speed coefficient used to control the sensitivity of dynamic response. A larger β value allows the filter to respond more quickly when the signal changes rapidly, reducing hysteresis, but it will also introduce more noise.

[0112] 3.2.3 Location Update

[0113] Final smooth position of the handle Iterative updates are performed using the following formula:

[0114]

[0115] The smoothing factor at the corresponding position ( )for:

[0116]

[0117] 3.3 Trajectory Prediction Based on LSTM

[0118] To further enhance the interactive experience and reduce system latency, this invention employs a Long Short-Term Memory (LSTM) network to perform short-term prediction of the handle trajectory, predicting the position in the next few frames.

[0119] 3.3.1 Data Input

[0120] With length as The historical window is used as input, and the features of each frame are... This is a combination of smooth position and smooth velocity. The input feature vector is expanded as follows:

[0121]

[0122] Build length is Historical sliding window sequence :

[0123]

[0124] 3.3.2 Trajectory Prediction

[0125] Will Input a pre-trained LSTM neural network model, and the model outputs the future... Predicted position sequence of frames :

[0126]

[0127] 3.3.3 Location Fusion

[0128] After completing the prediction, the system will extract the predicted sequence. Select a time step Then, the current smoothed position and the predicted position are linearly fused to obtain the fused position of the handle:

[0129]

[0130] in It is determined by a pre-set fixed cutoff frequency. The result is obtained through the following formula:

[0131]

[0132] Specifically, the system dynamically adjusts the prediction weights based on the current controller speed. When the smooth speed modulus of the handle exceeds the preset speed threshold, the algorithm calculates... As the value approaches 1, the system places greater trust in the predicted location. To compensate for delay; when the smooth speed modulus of the handle is less than or equal to the preset speed threshold, the algorithm calculates... As the value approaches zero, the system places greater trust in the current location. This is to prevent jitter caused by prediction noise.

[0133] 3.4 Ray Construction and Target Locking

[0134] 3.4.1 Ray Structure

[0135] In addition to position, the wireless controller also obtains a unit vector of the pointing direction through attitude calculation. ):

[0136]

[0137] Use the fusion location as the ray origin. ):

[0138]

[0139] Handle Ray ( () is represented in parameter form:

[0140]

[0141] Assume the standard form of the interaction plane π is:

[0142]

[0143] Where n is the plane normal vector, d is a constant, and x is the coordinate vector of any point in three-dimensional space.

[0144] The intersection of a ray and a plane satisfies:

[0145]

[0146] The intersection point parameters are obtained:

[0147]

[0148] 3.4.2 Target Selection

[0149] Suppose that the minimum distance between an object i and the predicted ray is less than a set threshold ε for M consecutive frames:

[0150]

[0151] Then it is determined that the user has a directional intention towards target i.

[0152] At this point, the cursor position on the screen is moved from the intersection of the planes ( Smoothly adsorbed to the target center point :

[0153]

[0154] The parameter γ controls the adsorption intensity. Generally, γ is taken as [0.3, 0.7]. The larger γ is, the faster the adsorption rate; the smaller γ is, the smoother the adsorption process.

[0155] 4. Test Environment and Test Plan

[0156] 4.1 Test Environment

[0157] Hardware: Standard wireless controller (sampling frequency 120Hz), VR headset (latency approximately 20ms), high-performance computer (CPU i7-12700H, GPU RTX 3060).

[0158] Test scenarios: 1) Static aiming scenario; 2) Dynamic tracking scenario; 3) Dense target scenario;

[0159] Comparison objects: traditional fixed-parameter low-pass filtering method (LPF), Kalman filtering method (KF), and direct processing method without filtering (Direct).

[0160] 4.2 Test Indicators

[0161] Jitter amplitude: Standard deviation of cursor position fluctuation over 10 seconds in a static state (handle stationary) (unit: mm).

[0162] Delay time: The time difference between the controller action and the virtual ray response (unit: ms);

[0163] Target selection accuracy: The ratio (%) of the number of times a target is successfully selected within a specified time (30 seconds) to the total number of attempts;

[0164] False selection rate: In a dense target scene, the ratio (%) of the number of times a non-target object is selected to the total number of selections.

[0165] 4.3 Test Results and Analysis

[0166] like Figure 2 As shown, Figure 2 This is a comparison chart of test metrics for different interaction methods, from... Figure 2 The following conclusions can be drawn:

[0167] 1. Jitter suppression effect: The jitter amplitude of this invention is only 0.73mm, which is 61.4% lower than that of traditional low-pass filtering and 53.2% lower than that of Kalman filtering. It significantly suppresses the fluctuations caused by wireless signal noise and physiological hand tremors, and greatly improves the stability of the cursor in static scenes.

[0168] 2. Delay compensation effect: The delay time of this invention is reduced to 8.5ms, which is 66.4% lower than the average of the prior art. It effectively offsets the inherent delay of sensor acquisition, attitude calculation and screen rendering, and avoids the problem of missing the target caused by spatiotemporal misalignment in dynamic tracking scenes.

[0169] 3. Target selection accuracy: For ultra-small targets of 5mm, the accuracy of this invention reaches 91.2%, which is 38.8% higher than the existing best Kalman filter; the misselection rate in dense target scenes is only 3.7%, which is more than 80% lower than traditional methods, solving the pain point of difficulty in selecting small and dense targets;

[0170] 4. Comprehensive adaptability: The present invention maintains excellent performance in different target sizes and different motion speeds, while existing technologies have obvious shortcomings (such as the accuracy of low-pass filtering decreases in high-speed scenarios and Kalman filtering is not suitable for ultra-small targets), which verifies the universality of the present invention.

[0171] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Based on the technical essence of the present invention, any simple modifications, equivalent substitutions, and improvements made to the above embodiments within the spirit and principles of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A high-precision handle interaction method based on adaptive filtering and deep learning, characterized in that, Includes the following steps: S1. Coordinate Transformation and Input Preprocessing: Obtain the original 3D observation position of the wireless handle in the tracking coordinate system in the discrete time domain, and map the original 3D observation position to the target model coordinate system through a pre-calibrated rigid body transformation matrix; S2. Adaptive filtering: The instantaneous velocity of the handle is estimated based on the adjacent sampling points. The instantaneous velocity is smoothed by low-pass filtering to obtain the smoothed velocity. The cutoff frequency of the position filter is dynamically adjusted according to the magnitude of the smoothed velocity, and then the current smoothed position of the handle is updated. S3, LSTM trajectory prediction: Construct a feature sequence containing smoothed positions and smoothed velocities from multiple historical frames, input the feature sequence into a pre-trained LSTM neural network model, and output a predicted position sequence for multiple future frames; S4. Position Fusion: Dynamically adjust the prediction weights based on the current controller speed, and linearly fuse the current smooth position with the predicted position at the corresponding delay time step to obtain the fused position; S5. Ray Construction and Target Locking: Starting from the fusion position, a virtual ray is constructed using the unit vector of the handle's pointing direction. If the distance between the target object and the virtual ray continuously meets the set threshold, the magnetic locking logic is triggered, smoothly attracting the cursor to the center of the target object, thus achieving high-precision target selection.

2. The high-precision handle interaction method based on adaptive filtering and deep learning according to claim 1, characterized in that, The coordinate transformation and input preprocessing in step S1 specifically include: The original handle point at each time step is expanded into a homogeneous vector. The rigid body transformation matrix includes a 3×3 rotation matrix and a 1×3 translation vector. The homogeneous vector is transformed by the rigid body transformation matrix to obtain the uniform coordinate position in the target model coordinate system.

3. The high-precision handle interaction method based on adaptive filtering and deep learning according to claim 1, characterized in that, The adaptive filtering process in step S2 specifically includes: The adaptive filtering is a One Euro filter; The instantaneous velocity is calculated based on the position difference between adjacent sampling points and the sampling period. The instantaneous velocity is then smoothed using a fixed cutoff frequency to obtain a smoothed velocity. Based on the preset minimum cutoff frequency, speed coefficient, and smoothing speed modulus, an adaptive cutoff frequency is calculated. At low speeds, the cutoff frequency is reduced to enhance the smoothing effect, while at high speeds, the cutoff frequency is increased to reduce latency. The position smoothing factor is calculated based on the adaptive cutoff frequency and the sampling frequency, and the current smoothed position is obtained through iterative updates.

4. The high-precision handle interaction method based on adaptive filtering and deep learning according to claim 1, characterized in that, The LSTM trajectory prediction in step S3 specifically includes: The feature sequence consists of continuous frame features of a length equal to a preset history window, and each frame feature is a combination of the smoothed position and smoothed speed at the corresponding moment. After being pre-trained, the LSTM neural network model receives feature sequences and outputs a predicted position sequence for the next preset number of frames. The predicted position sequence reflects the short-term movement trend of the handle.

5. The high-precision handle interaction method based on adaptive filtering and deep learning according to claim 1, characterized in that, The position fusion prediction weight adjustment logic in step S4 is as follows: when the handle speed is ≥ 0.4m / s, the prediction weight value tends to 1, and the system adopts the predicted position; when the handle speed is < 0.03m / s, the prediction weight value tends to 0, and the system adopts the current smooth position; when the handle speed is ≥ 0.03m / s and < 0.4m / s, the prediction weight changes continuously and linearly with the handle speed.

6. The high-precision handle interaction method based on adaptive filtering and deep learning according to claim 1, characterized in that, The ray construction in step S5 specifically includes: The virtual ray starts at the fusion position and takes the unit pointing vector obtained from the handle posture calculation as its direction, and is represented in parametric form. The point of action of the virtual ray on the interaction plane is obtained by solving the intersection equation of the virtual ray and the interaction plane.

7. The high-precision handle interaction method based on adaptive filtering and deep learning according to claim 1, characterized in that, Target locking in step S5 specifically includes: If the minimum distance between the target object and the virtual ray is less than a set threshold within a consecutive preset number of frames, it is determined that the user intends to point at the target object; the cursor is smoothly attracted from the point of action of the ray to the center of the target object by controlling the preset adsorption intensity parameter, so as to achieve stable locking.

8. A high-precision handle interaction method based on adaptive filtering and deep learning according to claim 1, characterized in that, The operating parameters in the discrete time domain satisfy the following: the sampling frequency is the reciprocal of the sampling period, and the system periodically acquires the original three-dimensional observation position of the handle according to the sampling period.

Images