A method and system for full-motor fusion positioning based on XR controllers

Abstract

This invention relates to a method and system for full-motion fusion positioning based on XR handheld devices, belonging to the field of visual positioning technology. It solves the problems of insufficient positioning accuracy and poor coordination with XR head-mounted displays in existing full-motion devices. The method includes: returning the full-motion device to its origin in the full-motion device coordinate system; acquiring the positioning data of the left and right handheld devices of the XR device in real time; fixing the left and right handheld devices of the XR device to the full-motion device; obtaining the corresponding fusion strategy based on the confidence level in the positioning data of the left and right handheld devices; obtaining the fused pose of the left handheld device by executing the fusion strategy; and locating the pose of the full-motion device in the XR device coordinate system based on the pre-calibrated extrinsic pose of the full-motion device and the fused pose of the left handheld device. This achieves precise positioning of the full-motion device.

Description

Technical Field

[0001] This invention relates to the field of visual positioning technology, and in particular to a full-motor fusion positioning method and system based on an XR handle. Background Technology

[0002] Full-motion flight simulators are core equipment for the retraining of civil aviation pilots. They mainly consist of a simulated cockpit, motion system, visual system, computer system, and instructor control console. Among these, the environmental visualization presented by the visual system directly affects the realism of the training. With the development of XR technology, its application in flight training enhances the immersive experience.

[0003] However, existing technologies have the following key shortcomings:

[0004] Some existing full-motion simulators do not support positioning output, which cannot meet the training scenario requirements of linkage with external interactive devices; others, although they have self-calculation positioning function, generally suffer from long-term error accumulation, which leads to a continuous decline in position simulation accuracy during training.

[0005] When the full-motion simulator and the XR head-mounted display are linked, the timestamps of the two devices are not synchronized, which directly leads to the inability to unify their positioning coordinate systems.

[0006] Existing solutions, such as deploying additional base station equipment like Optitrack or LightHouse, can achieve unified positioning, but they are costly, require additional space for base stations, and are prone to interference between base stations, which seriously hinders the large-scale deployment of full-motion simulators. Summary of the Invention

[0007] Based on the above analysis, the present invention aims to provide a full-motion fusion positioning method and system based on an XR controller, in order to solve the problems of insufficient positioning accuracy and poor coordination with XR headsets in existing full-motion systems.

[0008] On one hand, embodiments of the present invention provide a full-motion fusion positioning method based on an XR controller, comprising the following steps:

[0009] After the motor is returned to the origin of the motor coordinate system, the positioning data of the left and right handles of the XR device are acquired in real time; the left and right handles of the XR device are fixed on the motor.

[0010] Based on the confidence levels in the positioning data of the left and right controllers, the corresponding fusion strategy is obtained, and the fused pose of the left controller is obtained by executing the fusion strategy.

[0011] Based on the pre-calibrated extrinsic pose of the full motor and the fused pose of the left handle, the pose of the full motor in the XR device coordinate system is determined.

[0012] Based on further improvements to the above method, a corresponding fusion strategy is obtained according to the confidence levels in the positioning data of the left and right controllers, including:

[0013] When the maximum confidence score of the left and right hands is less than the minimum confidence score threshold, it is determined that the two hands are in failure, and the fusion strategy is the two hands failure localization process.

[0014] When the maximum confidence score of the left and right hands is greater than or equal to the minimum confidence score threshold, it is determined that at least one hands is valid, and the fusion strategy is a dual-hands fusion positioning process.

[0015] Based on further improvements to the above method, a dual-handle failure location process is executed, including:

[0016] Set the positioning position to the position at the last moment before the confidence of the left handle drops to the minimum confidence threshold; update the positioning attitude to the attitude in the positioning data of the left handle at the current moment;

[0017] Based on the positioning position and positioning attitude, the fused pose of the left handle is obtained.

[0018] Based on further improvements to the above method, a dual-handle fusion positioning process is executed, including:

[0019] The extended Kalman filter algorithm is used to fuse the predicted data of the left hand controller with the observation data of the right hand controller converted to the left hand controller to obtain the fused pose of the left hand controller.

[0020] Based on the above method, a further improvement is made to the extended Kalman filter algorithm, which includes:

[0021] Based on the state vector updated at the previous time step, obtain the predicted value of the state vector at the current time step; based on the predefined state transition matrix and process noise matrix, predict the state covariance updated at the previous time step, and obtain the predicted value of the state covariance at the current time step.

[0022] The pose in the positioning data of the right handle at the current moment is converted to the left handle to obtain the observation pose, and then the observation vector at the current moment is obtained.

[0023] Based on the observation matrix, observation noise matrix, current observation vector, state vector prediction, and state covariance prediction, the current observation error and Kalman gain are calculated, and then the current state vector prediction and state covariance prediction are updated to obtain the fused pose of the left handle at the current time.

[0024] Based on the further improvement of the above method, the process noise matrix is ​​a diagonal matrix, whose diagonal elements correspond to the noise parameters of the position, rotation and speed of the left handle, respectively.

[0025] Based on the further improvement of the above method, the observation noise matrix is ​​a diagonal matrix, whose diagonal elements correspond to the noise parameters of the position and rotation of the right handle, respectively.

[0026] Based on a further improvement of the above method, the noise parameters of the position and rotation of the right handle are set according to the inverse ratio of the confidence level of the right handle.

[0027] Based on the further improvement of the above method, the state vector includes: three-dimensional position, three-dimensional rotation vector and three-dimensional velocity; the three-dimensional position and three-dimensional rotation vector are extracted from the state vector updated at the current moment and converted into the fused pose of the left handle at the current moment.

[0028] On the other hand, embodiments of the present invention provide a full-motion fusion positioning system based on an XR handle, comprising:

[0029] The initial calibration module is used to acquire the positioning data of the left and right handles of the XR device in real time after the entire motor is returned to the origin; the left and right handles of the XR device are fixed on the entire motor.

[0030] The controller pose fusion module is used to obtain the corresponding fusion strategy based on the confidence level in the positioning data of the left and right controllers, and to obtain the fused pose of the left controller by executing the fusion strategy.

[0031] The full motor positioning module is used to locate the pose of the full motor in the XR device coordinate system based on the pre-calibrated extrinsic pose of the full motor and the fused pose of the left handle.

[0032] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0033] 1. By using the left and right handles built into the XR device as positioning sources and fixing them to the motor, the fusion positioning of the motor is realized. This not only enables the linkage between the motor and the XR device, but also eliminates the need to deploy additional base station positioning equipment, avoiding potential signal interference problems between base stations. This allows for large-scale deployment of the motor without increasing costs.

[0034] 2. By introducing a confidence judgment mechanism and two fusion strategies, the system intelligently responds to abnormal situations such as the handle being temporarily obscured or removed from the field of view. When both handles fail, the system minimizes pose drift and maintains the availability of the training system by locking the position and updating only the pose. When at least one handle is effective, the system adaptively adjusts the observation noise through the extended Kalman filter algorithm to dynamically optimize the fusion effect, ensuring the continuity and stability of the training process.

[0035] 3. By fusing the positioning data of two spatially separated handles and using the extended Kalman filter algorithm for optimal estimation, the noise and fluctuation of a single sensor are effectively suppressed, making the positioning results more stable and accurate than those from a single data source. Moreover, the algorithm corrects errors in real time, solving the problem of error accumulation caused by long-term integration.

[0036] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0037] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0038] Figure 1 This is a flowchart of a full-motor fusion positioning method based on an XR handle in Embodiment 1 of the present invention;

[0039] Figure 2 This is a schematic diagram of the initial calibration of the XR device and the engine in Embodiment 1 of the present invention. Detailed Implementation

[0040] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0041] Example 1

[0042] A specific embodiment of the present invention discloses a full-motor fusion positioning method based on XR controllers. Its core lies in utilizing the positioning data of the left and right XR controllers fixed to the full motor, and through a specific fusion strategy, to locate the high-precision, high-frequency pose of the full motor in the XR device coordinate system. The XR device then renders and displays the corresponding virtual scene image in real time based on the pose of the full motor. For example... Figure 1 As shown, it includes the following steps:

[0043] S1. After returning the entire motor to the origin of the entire motor coordinate system, acquire the positioning data of the left and right handles of the XR device in real time; the left and right handles of the XR device are fixed on the entire motor.

[0044] S2. Based on the confidence levels in the positioning data of the left and right controllers, obtain the corresponding fusion strategy, and obtain the fused pose of the left controller by executing the fusion strategy.

[0045] S3. Based on the pre-calibrated extrinsic pose of the full motor and the fused pose of the left handle, locate the pose of the full motor in the XR equipment coordinate system.

[0046] It should be noted that the full name of the "full-motion simulator" in this embodiment is "full-motion flight simulator," and the XR equipment includes: an XR head-up display, a left controller, and a right controller.

[0047] During implementation, first fix the left and right handles that come with the XR headset to the motor. For example, securely install them in the center of the motor using a rigid clamp. Just make sure they form a rigid body with the motor and that the left and right handles are within the field of view of the XR headset's positioning camera during training.

[0048] The external parameter position of the left handle relative to the entire motor is obtained in advance through design drawings or precise measurements. The external parameter pose of the right handle relative to the entire motor .

[0049] Turn on the motor and XR equipment. The operator, wearing the XR headset, begins training by manipulating the joysticks on the motor. The left and right joysticks move together with the motor. First, in step S1, as... Figure 2 As shown, the entire motor is returned to the origin of its own coordinate system. The XR head-up display faces the left and right handles, ensuring that the left and right handles are completely within the field of view of the XR head-up display's positioning camera. This ensures that the left and right handles can be stably and accurately tracked by the XR head-up display's positioning system. The positioning data of the left and right handles of the XR device are obtained in real time through the positioning interface provided by the XR device.

[0050] Specifically, the positioning data for the left handle includes: pose. ,speed and confidence level The positioning data for the right handle includes: pose. and confidence level Pose includes position and orientation. Position is generally represented by a three-dimensional coordinate vector, while orientation is generally represented by a rotation matrix, Euler angles, or quaternions.

[0051] The position of the right handle relative to the left handle is then calculated using the following formula. :

[0052] ,

[0053] in, This indicates the matrix inversion operation.

[0054] After the initial calibration in step S1 is completed, the pose fusion of the left and right handles is achieved in step S2 during real-time operation.

[0055] It should be noted that when the left and right controllers are not within the camera's field of view of the XR device, or when there is interference such as obstruction, the confidence level of the controllers output by the XR device will decrease. In order to intelligently deal with abnormal situations such as the controllers being temporarily obstructed or moved out of the field of view, this embodiment subdivides step S2. By monitoring the comparison result of the confidence level of the left and right controllers with the preset minimum confidence level threshold in step S20, the system identifies the situation where both controllers are ineffective or at least one controller is effective, obtains different fusion strategies, and executes the corresponding fusion strategy steps S21 or S22 to ensure the continuity and stability of the training process.

[0056] S20, Obtain the fusion strategy.

[0057] Specifically, based on the confidence levels in the positioning data of the left and right controllers, a corresponding fusion strategy is obtained, including:

[0058] When the maximum confidence score of the left and right hands is less than the minimum confidence score threshold, it is determined that the dual handset has failed. The fusion strategy is the dual handset failure localization process, and step S21 is executed.

[0059] When the maximum confidence score of the left and right hands is greater than or equal to the minimum confidence score threshold, it is determined that at least one hands is valid, the fusion strategy is the dual-hands fusion positioning process, and step S22 is executed.

[0060] S21. Execute the dual-handle failure location procedure.

[0061] When dual-handle positioning fails, to avoid complete loss of positioning data and resulting in "drift" of the entire motor's pose, perform the following steps:

[0062] Set the positioning position to the position at the last moment before the confidence of the left handle drops to the minimum confidence threshold; that is, during the current moment and subsequent failures, keep the position of the left handle fixed at the position at the last valid moment.

[0063] At the same time, the positioning attitude is updated to the attitude in the positioning data of the left handle at the current moment; that is, the attitude of the left handle is updated normally in real time.

[0064] Therefore, based on the fixed positioning position and the real-time positioning attitude, the fused pose of the left handle at the current moment is obtained.

[0065] It should be noted that the dual-handle failure localization process adopts a strategy of locking the position and updating only the pose to minimize pose drift and maintain the availability of the training system.

[0066] S22. Execute the dual-handle fusion positioning process.

[0067] When one or both controllers are active, the extended Kalman filter algorithm is used to fuse the predicted data of the left controller with the observation data of the right controller converted to the left controller to obtain the fused pose of the left controller.

[0068] It should be noted that the real-time positioning data of the left hand controller is used as the prediction source data, and the real-time positioning data of the right hand controller is used as the observation source data.

[0069] Before executing the extended Kalman filter algorithm, initialize the state covariance matrix, process noise matrix, observation noise matrix, state transition matrix, and observation matrix, and define the state vector.

[0070] Among them, the state covariance matrix and state transition matrix All are initialized to a 9×9 identity matrix; in this embodiment, the time interval is particularly short and the change of the state vector is also very small, so the state transition matrix is ​​approximately an identity matrix.

[0071] Process noise matrix It is a 9×9 diagonal matrix used to describe the measurement noise of the left handle's positioning data; only the diagonal elements are non-zero, and the rest are zero; its diagonal elements correspond to the noise parameters of the left handle's position, rotation (i.e., attitude) and speed, respectively.

[0072] Specifically, the first three elements on the diagonal are the noise of the three-dimensional position of the left handle, which is initialized to 0.001 for example; the middle three elements on the diagonal are the noise of the three-dimensional rotation vector of the left handle, which is initialized to 0.00001 for example; and the last three elements on the diagonal are the noise of the three-dimensional velocity of the left handle, which is initialized to 0.0001 for example.

[0073] Observation noise matrix It is a 6×6 diagonal matrix used to describe the measurement noise of the right handle's positioning data; only the diagonal elements are non-zero, and the rest are zero; its diagonal elements correspond to the noise parameters of the right handle's position and rotation, respectively, and these noise parameters are based on the confidence level of the right handle. The inverse ratio is obtained by setting it.

[0074] Specifically, the first three elements on the diagonal are noise representing the three-dimensional position of the right handle, initialized as follows: The last three elements on the diagonal are noise from the right handle's 3D rotation vector, initialized to... .

[0075] Observation matrix It is a 6×9 matrix. That is, it consists of a 6×6 identity matrix. and a 6×3 zero matrix The system consists of an identity matrix used to obtain the position and rotation vector of the right handle, and a zero matrix used to represent the velocity of the unobserved right handle.

[0076] State vector It is a 9×1 vector. ,in, This indicates the three-dimensional position of the left handle. This represents the three-dimensional rotation vector of the left handle. This represents the three-dimensional velocity of the left handle. The initial value of the state vector. It is based on the pose of the left handle in the positioning data at the initial moment. and speed And obtained, among which correspond The position part in the middle, Through the It is obtained by transforming the rotation matrix part in the image; Corresponding speed .

[0077] Furthermore, the process of executing the extended Kalman filter algorithm, driven by the left handle and corrected by the right handle, includes:

[0078] ①Prediction phase

[0079] Based on the updated state vector from the previous time step, the predicted state vector value for the current time step is obtained, as shown in the following formula:

[0080] ,

[0081] in, express The predicted state vector value at time t. express The state vector updated at each time step.

[0082] Based on the predefined state transition matrix and process noise matrix, the updated state covariance of the previous time step is predicted to obtain the predicted state covariance value of the current time step, as shown in the following formula:

[0083] ,

[0084] in, and They represent The predicted state covariance at time t and State covariance at time t, This represents the matrix transpose operation.

[0085] ②Observation conversion

[0086] Based on the initial calibrated position of the right handle relative to the left handle The pose of the right handle in the current positioning data. Switch to the left handle, that is, use the right handle pose to calculate the corresponding left handle pose, and obtain the observation pose. The formula is shown below:

[0087] .

[0088] Furthermore, from Extract observation location and observation rotation vector To form the observation vector at the current moment , is a 6×1 vector.

[0089] ③ Update Phase

[0090] Based on the observation matrix, the current observation vector, and the predicted state vector, the observation error at the current moment is calculated using the following formula:

[0091] ,

[0092] in, express The observation error at a given time.

[0093] The Kalman gain at the current time is calculated based on the observation matrix, the observation noise matrix, and the predicted state covariance at the current time, as shown in the following formula:

[0094] ,

[0095] in, express Kalman gain at time step.

[0096] Based on the observation error and Kalman gain at the current moment, update the predicted values ​​of the state vector and state covariance at the current moment to obtain the updated state vector and state covariance at the current moment, which are used in the next prediction stage. The formula is shown below:

[0097] ,

[0098] .

[0099] Simultaneously, the 3D position and 3D rotation vectors are extracted from the updated state vector at the current moment and converted into the fused pose of the left handle at the current moment. .

[0100] It should be noted that the dual-handle fusion positioning process effectively suppresses noise and fluctuations from a single sensor by fusing positioning data from two spatially separate handles and using the EKF algorithm for optimal estimation. This results in more stable and accurate positioning results than from a single data source. Furthermore, by adaptively adjusting observation noise and dynamically optimizing the fusion strategy, the continuity and stability of the training process are ensured.

[0101] After obtaining the fused pose of the left handle at each moment through step S2, it is compared with the pre-calibrated extrinsic pose of the entire motor. Multiplying the inverse of the product yields the pose of the entire motor in the XR device coordinate system, as shown in the following formula:

[0102] ,

[0103] in, This indicates the pose of the entire motor in the XR equipment coordinate system.

[0104] Finally, the pose of the entire motion unit in the XR device coordinate system at each moment is sent to the rendering engine of the XR device in real time. The rendering engine adjusts the trainer's perspective in the virtual world based on this pose, renders the corresponding virtual scene image in real time and displays it in the XR headset, thereby achieving synchronization between the visual image and the physical movement of the entire motion unit, which greatly enhances the immersion and realism of the training.

[0105] Compared with existing technologies, this embodiment provides a full-motor fusion localization method based on XR handles. Utilizing the left and right handles of the XR device as localization sources, fixed to the full motor, it achieves full-motor fusion localization. This not only enables linkage between the full motor and the XR device but also eliminates the need for additional base station localization equipment, avoiding potential signal interference between base stations and enabling large-scale deployment of the full motor without increasing costs. By introducing a confidence judgment mechanism and two fusion strategies, it intelligently handles abnormal situations such as handles being temporarily obstructed or removed from the field of view. When both handles fail, a strategy of locking the position and updating only the attitude minimizes pose drift and maintains the availability of the training system. When at least one handle is effective, an extended Kalman filter algorithm adaptively adjusts the observation noise, dynamically optimizing the fusion effect and ensuring the continuity and stability of the training process. By fusing the localization data of two spatially separated handles and using the extended Kalman filter algorithm for optimal estimation, it effectively suppresses noise and fluctuations from a single sensor, making the localization results more stable and accurate than those from a single data source. Furthermore, real-time algorithm correction solves the problem of error accumulation caused by long-term integration.

[0106] Example 2

[0107] Another embodiment of the present invention discloses a full-motor fusion positioning system based on an XR handle, thereby implementing the full-motor fusion positioning method based on an XR handle in Embodiment 1. The specific implementation of each module is described in the corresponding description in Embodiment 1. The system includes:

[0108] The initial calibration module is used to acquire the positioning data of the left and right handles of the XR device in real time after the entire motor is returned to the origin; the left and right handles of the XR device are fixed on the entire motor.

[0109] The controller pose fusion module is used to obtain the corresponding fusion strategy based on the confidence level in the positioning data of the left and right controllers, and to obtain the fused pose of the left controller by executing the fusion strategy.

[0110] The full motor positioning module is used to locate the pose of the full motor in the XR device coordinate system based on the pre-calibrated extrinsic pose of the full motor and the fused pose of the left handle.

[0111] Since the XR handle-based full-motor fusion positioning system of this embodiment and the aforementioned XR handle-based full-motor fusion positioning method are related and can be mutually referenced, this description is redundant and will not be repeated here. Because this system embodiment shares the same principle as the above method embodiment, it also possesses the corresponding technical effects of the above method embodiment.

[0112] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0113] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A full-motor fusion positioning method based on an XR handle, characterized in that, Includes the following steps: After the motor is returned to the origin of the motor coordinate system, the positioning data of the left and right handles of the XR device are acquired in real time; the left and right handles of the XR device are fixed on the motor. Based on the confidence levels in the positioning data of the left and right controllers, a corresponding fusion strategy is obtained, and the fused pose of the left controller is obtained by executing the fusion strategy. Specifically, when the maximum confidence level of the left and right controllers is greater than or equal to the minimum confidence threshold, it is determined that at least one controller is valid, and the fusion strategy is a dual-controller fusion positioning process. The dual-controller fusion positioning process includes: using an extended Kalman filter algorithm to fuse the predicted data of the left controller with the observation data of the right controller converted to the left controller to obtain the fused pose of the left controller; the observation noise matrix in the extended Kalman filter algorithm is a diagonal matrix, whose diagonal elements correspond to the noise parameters of the right controller's position and rotation, respectively. The noise parameters of the right controller's position and rotation are set according to the inverse proportion of the right controller's confidence level. Based on the pre-calibrated extrinsic pose of the full motor and the fused pose of the left handle, the pose of the full motor in the XR device coordinate system is located.

2. The all-motor fusion positioning method based on an XR handle according to claim 1, characterized in that, The step of obtaining the corresponding fusion strategy based on the confidence levels in the positioning data of the left and right controllers further includes: When the maximum confidence score of the left and right hands is less than the minimum confidence score threshold, it is determined that the two hands are in failure, and the fusion strategy is the two hands failure localization process.

3. The full-motor fusion positioning method based on an XR handle according to claim 2, characterized in that, The process of performing the dual-handle failure location includes: Set the positioning position to the position at the last moment before the confidence of the left handle drops to the minimum confidence threshold; update the positioning attitude to the attitude in the positioning data of the left handle at the current moment; Based on the positioning position and positioning posture, the fused posture of the left handle is obtained.

4. The full-motor fusion positioning method based on an XR handle according to claim 1, characterized in that, The extended Kalman filter algorithm is employed, including: Based on the state vector updated at the previous time step, obtain the predicted value of the state vector at the current time step; based on the predefined state transition matrix and process noise matrix, predict the state covariance updated at the previous time step, and obtain the predicted value of the state covariance at the current time step. The pose in the positioning data of the right handle at the current moment is converted to the left handle to obtain the observation pose, and then the observation vector at the current moment is obtained. Based on the observation matrix, observation noise matrix, current observation vector, state vector prediction, and state covariance prediction, the current observation error and Kalman gain are calculated, and then the current state vector prediction and state covariance prediction are updated to obtain the fused pose of the left handle at the current time.

5. The full-motor fusion positioning method based on an XR handle according to claim 4, characterized in that, The process noise matrix is ​​a diagonal matrix, whose diagonal elements correspond to the noise parameters of the position, rotation, and speed of the left handle, respectively.

6. The full-motor fusion positioning method based on an XR handle according to claim 4, characterized in that, The state vector includes: three-dimensional position, three-dimensional rotation vector, and three-dimensional velocity; the three-dimensional position and three-dimensional rotation vector are extracted from the state vector updated at the current moment and converted into the fused pose of the left handle at the current moment.

7. A full-motion fusion positioning system based on an XR controller, characterized in that, include: The initial calibration module is used to acquire the positioning data of the left and right handles of the XR device in real time after the entire motor is returned to its origin; the left and right handles of the XR device are fixed on the entire motor. The controller pose fusion module is used to obtain a corresponding fusion strategy based on the confidence levels in the positioning data of the left and right controllers, and to obtain the fused pose of the left controller by executing the fusion strategy. Specifically, when the maximum confidence level of the left and right controllers is greater than or equal to the minimum confidence threshold, it is determined that at least one controller is valid, and the fusion strategy is a dual-controller fusion positioning process. The dual-controller fusion positioning process includes: using an extended Kalman filter algorithm to fuse the predicted data of the left controller with the observation data of the right controller converted to the left controller to obtain the fused pose of the left controller; the observation noise matrix in the extended Kalman filter algorithm is a diagonal matrix, whose diagonal elements correspond to the noise parameters of the right controller's position and rotation, respectively. The noise parameters of the right controller's position and rotation are set according to the inverse proportion of the right controller's confidence level. The full motor positioning module is used to locate the pose of the full motor in the XR device coordinate system based on the pre-calibrated extrinsic pose of the full motor and the fused pose of the left handle.

Images