A positioning device for a full-motion flight simulator based on automatic calibration of time deviation

Abstract

This invention relates to a positioning device for a full-motion flight simulator based on automatic calibration of time deviation, belonging to the field of visual positioning technology. It solves the problem of inaccurate positioning caused by time asynchrony between the full-motion simulator and the XR device. The device includes: a left and right hand controller fixed to the full-motion flight simulator; an XR headset communicatively connected to the full-motion flight simulator and equipped with a positioning processing system, which includes: a hand controller positioning fusion module fusing positioning data from the left and right hand controllers to obtain fused positioning data for the left hand controller; a time alignment module calculating the estimated pose of the left hand controller based on the full-motion simulator's positioning data and calibrating the time deviation; a filtering fusion module fusing and filtering the time-aligned estimated pose of the left hand controller and the fused positioning data, and outputting an optimized pose for the left hand controller; and a full-motion simulator positioning module locating the pose of the full-motion simulator in the XR device coordinate system based on the fused positioning data or the optimized pose of the left hand controller. This achieves precise positioning of the full-motion simulator.

Description

Technical Field

[0001] This invention relates to the field of visual positioning technology, and in particular to a positioning device for a full-motion flight simulator based on automatic calibration of time deviation. Background Technology

[0002] Full-motion flight simulators are core equipment for retraining civil aviation pilots. They typically consist of a simulator cockpit, motion system, visual system, computer system, and instructor control console. Among these, the realism of the environmental visualization provided by the visual system is crucial to the training effect.

[0003] With the integration of extended reality technology into flight training, the existing technical solutions mainly include the following:

[0004] The first type is the positioning scheme provided by the full-motion flight simulator itself, but it generally suffers from the problem of error accumulation over time, which leads to a continuous decline in position simulation accuracy during long-term training, making it difficult to meet the standards of high-fidelity training.

[0005] The second category is when the full-motion flight simulator and XR equipment are linked. Since the two are independent systems, their equipment timestamps are not synchronized, which makes it impossible to fuse their positioning data under a unified spatiotemporal reference, that is, the coordinate system cannot be unified.

[0006] The third type relies on external base station equipment to achieve unified coordinate system positioning between XR equipment and the full-motion flight simulator. Existing technologies typically employ methods such as deploying high-precision external base stations like OptiTrack or LightHouse. While this approach achieves accurate positioning, it has significant drawbacks: First, the additional base station equipment is expensive, significantly increasing system costs and deployment complexity; second, base stations need to be installed on-site, occupying extra space, and signal interference may exist between multiple base stations. These problems severely hinder the low-cost, high-efficiency large-scale upgrade and deployment of full-motion flight simulators.

[0007] In addition, existing solutions that rely on XR controllers for positioning have the problem that positioning data is immediately lost when the controller goes out of the field of view of the XR head-mounted display positioning camera, resulting in a break in positioning continuity. If positioning is relied on both XR controllers and full-motion flight simulators, positioning fusion cannot be performed directly due to time misalignment. Summary of the Invention

[0008] Based on the above analysis, the present invention aims to provide a positioning device for a full-motion flight simulator based on automatic calibration of time deviation, in order to solve the problem of inaccurate positioning caused by the time asynchrony between the full-motion flight simulator and XR equipment.

[0009] This invention provides a positioning device for a full-motion flight simulator based on automatic time deviation calibration, comprising: an XR headset, a left handle, and a right handle; the left and right handles are fixed to the full-motion flight simulator, the XR headset is communicatively connected to the full-motion flight simulator, and a positioning processing system is deployed thereon; the positioning processing system includes:

[0010] The controller positioning fusion module is used to fuse the positioning data of the left and right controllers to obtain the fused positioning data of the left controller.

[0011] The time alignment module is used to calculate the estimated pose of the left hand controller based on the positioning data of the full-motion flight simulator, calibrate the time deviation between the estimated pose of the left hand controller and the pose in the fused positioning data of the left hand controller, and obtain the time-aligned estimated pose of the left hand controller based on the time deviation.

[0012] The filtering and fusion module is used to perform fusion filtering on the time-aligned estimated pose of the left hand controller and the fused positioning data of the left hand controller, and output the optimized pose of the left hand controller.

[0013] The full-motion positioning module is used to locate the pose of the full-motion flight simulator in the XR equipment coordinate system based on the fused positioning data of the left hand handle or the optimized pose of the left hand handle.

[0014] Based on further improvements to the above device, the positioning data includes: pose, confidence level, and velocity; the handle positioning fusion module acquires the fused positioning data of the left handle through the following steps:

[0015] Based on the relative calibration parameters of the left and right handles, the positioning data of the right handle at the current moment is converted to the left handle to obtain the backup data of the left handle at the current moment.

[0016] Based on the confidence levels of the left and right controllers at the current moment, obtain the positioning status of the left and right controllers.

[0017] Based on the comparison of the positioning status of the left and right controllers, the current left controller fused positioning data is obtained according to the backup data of the left controller at the current moment, the positioning data of the left controller at the current moment, or the fused positioning data of the left controller at the previous moment.

[0018] Based on further improvements to the above device, the current left handle fusion positioning data is obtained by acquiring the fusion attitude, fusion position, and fusion speed respectively; wherein, the fusion attitude is obtained by selecting the attitude with the larger positioning state from the backup data of the left handle at the current time and the positioning data of the left handle at the current time; the fusion position and fusion speed are obtained according to the fusion strategy corresponding to the comparison result.

[0019] Based on further improvements to the aforementioned device, in the time alignment module, the estimated pose of the left handle is calculated using the following formula based on the positioning data from the full-motion flight simulator:

[0020] ,

[0021] in, This indicates the estimated pose of the left handle. This indicates the world coordinate system of the full-motion flight simulator when it returns to its original position. In the world coordinate system of XR headset The lowered position; This represents the pose in the positioning data of a full-motion flight simulator; This indicates the pre-calibrated external parameter position and attitude of the left handle relative to the full-motion flight simulator.

[0022] Based on further improvements to the aforementioned device, the time alignment module calibrates the time deviation between the estimated pose of the left handle and the pose in the fused positioning data of the left handle, including:

[0023] The estimated pose of the left hand controller obtained in real time is sequentially placed into the first queue, and the fused positioning data of the left hand controller obtained in real time is sequentially placed into the second queue.

[0024] When the length of the first queue is greater than or equal to the length threshold, the first queue and the second queue respectively calculate their respective initial angular velocity sequences based on every two adjacent poses.

[0025] Interpolate the initial angular velocity sequences of the first and second queues respectively to obtain optimized angular velocity sequences with aligned timestamps and the same length;

[0026] The objective function is to minimize the difference between the two optimized angular velocity sequences after time deviation adjustment. The optimal time deviation is obtained by solving the objective function through an optimization algorithm.

[0027] Based on further improvements to the above-mentioned device, the objective function is expressed by the following formula:

[0028] ,

[0029] in, Indicates time deviation; This indicates the length of the angular velocity optimization sequence. This indicates that the optimized angular velocity sequence of the first queue is in angular velocity at time t; This indicates that the angular velocity optimization sequence of the second queue is in angular velocity at time t; The modulus representing the orientation quantity.

[0030] Based on further improvements to the above-mentioned device, the time alignment module further includes: when the length of the first queue is less than the length threshold, transmitting the fused positioning data of the left hand handle to the positioning module of the full-motion flight simulator to locate the pose of the full-motion flight simulator in the XR equipment coordinate system.

[0031] Based on further improvements to the above-mentioned device, the filtering and fusion module uses an extended Kalman filter to perform fusion filtering based on a uniform motion model, and then extracts the three-dimensional position and three-dimensional rotation vectors from the updated state vector and converts them to obtain the optimized pose of the left handle; the state vector includes: three-dimensional position, three-dimensional rotation vector and three-dimensional velocity.

[0032] Based on further improvements to the above device, the observation vector in the extended Kalman filter is derived from the left handle fused positioning data output by the handle positioning fusion module or the time-aligned estimated pose of the left handle output by the time alignment module.

[0033] Based on further improvements to the aforementioned device, the full-motion positioning module, using fused positioning data from the left hand handle or optimized pose of the left hand handle, determines the pose of the full-motion flight simulator in the XR equipment coordinate system using the following formula:

[0034] ,

[0035] in, This indicates the pose of the full-motion flight simulator in the XR equipment coordinate system. This indicates the fused pose in the left hand controller's fused positioning data, or the optimized pose of the left hand controller. This indicates the pre-calibrated external parameter position and attitude of the left handle relative to the full-motion flight simulator.

[0036] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0037] 1. It enables the linkage between the full-motion flight simulator and the XR head-mounted display, and eliminates the need to deploy additional base station positioning equipment, greatly reducing system costs and deployment complexity, which is conducive to the large-scale promotion and application of the full-motion flight simulator.

[0038] 2. An automatic time deviation calibration method based on the consistency of rigid body motion angular velocity is proposed. This method automatically and accurately calibrates and compensates for the time deviation between the full-motion flight simulator and the XR head-mounted display, solving the problem that existing heterogeneous data cannot be directly fused for positioning due to time asynchrony.

[0039] 3. By using the extended Kalman filter algorithm, the time-aligned positioning data of the full-motion flight simulator and the fused positioning data of the XR controller are deeply integrated. The advantages are complementary. It utilizes the smoothness of the positioning data of the full-motion flight simulator and combines the relative accuracy of the positioning of the XR controller. It effectively avoids positioning interruption caused by the failure of a single data source, and finally outputs a more stable and accurate pose of the full-motion flight simulator in the XR coordinate system.

[0040] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0041] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0042] Figure 1 This is a schematic diagram of a full-motion flight simulator positioning device based on automatic time deviation calibration in Embodiment 1 of the present invention;

[0043] Figure 2 This is a schematic diagram of the initial calibration of the XR equipment and the full-motion flight simulator in Embodiment 1 of the present invention. Detailed Implementation

[0044] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0045] A specific embodiment of the present invention discloses a positioning device for a full-motion flight simulator based on automatic time deviation calibration, such as... Figure 1 As shown, it includes: an XR headset, a left controller, and a right controller; the left and right controllers are fixed to the full-motion flight simulator, the XR headset is communicatively connected to the full-motion flight simulator, and a positioning processing system is deployed thereon; the positioning processing system includes:

[0046] The controller positioning fusion module is used to fuse the positioning data of the left and right controllers to obtain the fused positioning data of the left controller.

[0047] The time alignment module is used to calculate the estimated pose of the left hand controller based on the positioning data of the full-motion flight simulator, calibrate the time deviation between the estimated pose of the left hand controller and the pose in the fused positioning data of the left hand controller, and obtain the time-aligned estimated pose of the left hand controller based on the time deviation.

[0048] The filtering and fusion module is used to perform fusion filtering on the time-aligned estimated pose of the left hand controller and the fused positioning data of the left hand controller, and output the optimized pose of the left hand controller.

[0049] The full-motion positioning module is used to locate the pose of the full-motion flight simulator in the XR equipment coordinate system based on the fused positioning data of the left hand handle or the optimized pose of the left hand handle.

[0050] It should be noted that the XR equipment includes: an XR headset and its matching left and right handles; the left and right handles are fixed to the full-motion flight simulator, for example, by using rigid clamps to securely install them in the center of the full-motion flight simulator, as long as they form a rigid body with the full-motion flight simulator, and during training, the left and right handles can be simultaneously within the field of view of the XR headset's positioning camera.

[0051] The XR headset establishes a communication connection with the full-motion flight simulator via wired or wireless means to receive positioning data from the full-motion flight simulator. The positioning processing system deployed in the headset achieves high-precision and high-robust positioning of the full-motion flight simulator in the XR coordinate system through a series of collaborative software modules.

[0052] During implementation, in the initial calibration phase, the external parameter position and attitude of the left hand handle relative to the full-motion flight simulator are obtained in advance through design drawings or precision measurements. The right hand handle relative to the external parameter attitude of the full-motion flight simulator .

[0053] Turn on the full-motion flight simulator and XR equipment. The operator, wearing an XR headset, begins training by controlling the joysticks provided with the full-motion flight simulator. The left and right joysticks move together with the full-motion flight simulator. First, as... Figure 2 As shown, the full-motion flight simulator is returned to the origin of the full-motion flight simulator coordinate system. The XR head-up display is facing the left and right handles, so that the left and right handles are completely within the field of view of the XR head-up display positioning camera. This ensures that the left and right handles can be stably and accurately tracked by the positioning system of the XR head-up display. The positioning data of the left and right handles of the XR device are obtained in real time through the positioning interface provided by the XR device.

[0054] It should be noted that the positioning data includes: pose, confidence level, and velocity; specifically, the positioning data for the left controller includes: pose. ,speed and confidence level The positioning data for the right handle includes: pose. ,speed and confidence level Pose, in this context, includes position and orientation. Position is typically represented by a three-dimensional coordinate vector, while orientation is generally represented by a rotation matrix, Euler angles, or quaternions. That is: , , and These represent the positions of the left and right controllers in their respective poses. and These represent the poses of the left and right controllers, respectively.

[0055] The relative calibration parameters of the left and right handles, i.e., the position of the right handle relative to the left handle, are then calculated using the following formula. :

[0056] ,

[0057] in, This indicates the matrix inversion operation.

[0058] It should be noted that when returning the full-motion flight simulator to its coordinate system origin, the full-motion flight simulator coordinate system and the full-motion flight simulator world coordinate system are... If they coincide, the world coordinate system of the full-motion flight simulator can be calculated using the following formula. In the world coordinate system of XR headset Lower position :

[0059] ,

[0060] in, The pose in the positioning data of the full-motion flight simulator is obtained through the positioning interface provided by the full-motion flight simulator.

[0061] After the initial calibration is completed, when the positioning output frame rate of the full-motion flight simulator and the positioning output frame rate of the left and right hand controllers are greater than the set minimum frame rate threshold, such as 30Hz, the positioning processing system enters the real-time fusion positioning stage and completes the fusion positioning through the internal software module.

[0062] It should be noted that the controller positioning fusion module is centered on the left controller, and comprehensively utilizes the positioning information of both the left and right controllers to output a higher quality and more stable fused pose of the left controller, in order to cope with the positioning loss or accuracy reduction caused by the obstruction of a single controller.

[0063] Specifically, the controller positioning fusion module receives the positioning data of the left and right controllers at various times in real time. To facilitate fusion, for each moment, the module first uses the relative calibration parameters of the left and right controllers obtained from the initial calibration. The positioning data of the right controller at the current moment is converted to that of the left controller to obtain the backup data of the left controller at the current moment, including: backup pose. and backup speed The calculation formula is as follows:

[0064] ,

[0065] in, Indicates the relative calibration parameters of the left and right handles Rotation parameters in the configuration; alternative pose Including: backup locations and standby posture .

[0066] Furthermore, based on the confidence levels of the left and right controllers at the current moment, the positioning status of the left and right controllers is obtained, including:

[0067] If the confidence level is greater than or equal to the preset high confidence threshold, the positioning status is good quality.

[0068] If the confidence level is less than or equal to the preset low confidence threshold, the positioning status is poor quality.

[0069] Otherwise, the positioning status is "average quality".

[0070] It should be noted that, among the three levels of positioning status, good quality > average quality > poor quality.

[0071] For example, the high confidence threshold is set to 0.8 and the low confidence threshold is set to 0.3.

[0072] Furthermore, based on the comparison results of the positioning status of the left and right controllers, the current left controller fusion positioning data is obtained according to the current backup data of the left controller, the current positioning data of the left controller, or the previous left controller fusion positioning data.

[0073] It should be noted that the current left handle fusion positioning data includes: fused pose. and fusion speed Among them, fusion pose Including: fusion location and fusion posture .

[0074] The fused attitude is obtained by selecting the attitude with the larger positioning state from the backup data of the left controller at the current moment and the positioning data of the left controller at the current moment.

[0075] Specifically, if the positioning state of the left controller is greater than or equal to the positioning state of the right controller, then the attitude from the positioning data of the left controller is taken. Otherwise, take the attitude from the backup data of the left handle at the current moment, i.e. .

[0076] The fused position and fused speed are obtained based on the fusion strategy corresponding to the comparison results; wherein, the fusion strategy includes: using the position and speed in the fused positioning data of the left hand at the current moment; using the position and speed in the backup data of the left hand at the current moment; using the position and speed in the fused positioning data of the left hand at the previous moment; and weighting the position and speed in the positioning data of the left hand at the current moment and the backup data of the left hand at the current moment according to the confidence of the left hand and the right hand at the current moment.

[0077] Specifically, if the positioning status of the left controller is higher than that of the right controller, or if the positioning status of the left controller is equal to that of the right controller and the positioning quality is good, then the positioning data of the left controller at the current moment will be used as the fused positioning data of the left controller at the current moment. The formula is as follows:

[0078] .

[0079] If the positioning status of the left controller is lower than that of the right controller, then the current backup data of the left controller will be used as the current fused positioning data of the left controller. The formula is as follows:

[0080] .

[0081] If the positioning state of the left controller is equal to that of the right controller and the positioning state is of poor quality, then the fused positioning data of the left controller from the previous moment will be used as the fused positioning data of the left controller at the current moment. The formula is as follows:

[0082] ,

[0083] in, and This indicates the position and velocity of the left handle in the fused positioning data at the previous moment.

[0084] Otherwise, the backup data and positioning data of the left hand controller at the current moment are weighted and fused to obtain the fused positioning data of the left hand controller at the current moment; where the weights of the left and right hand controllers are obtained according to their respective confidence ratios; the formula is expressed as:

[0085] ,

[0086] in, and These are the weights of the left and right controllers, respectively.

[0087] Finally, the controller positioning fusion module outputs the left controller fused positioning data (fused pose). and fusion speed This provides high-quality handle motion estimation for the subsequent time alignment module.

[0088] It should be noted that the time alignment module is used to solve the problem of asynchronous positioning time between the full-motion flight simulator and the controller, which is a prerequisite for achieving high-precision heterogeneous data fusion.

[0089] First, based on the positioning data from the full-motion flight simulator, the estimated pose of the left handgrip is calculated using the following formula:

[0090] ,

[0091] in, This indicates the estimated pose of the left handle, including its position. and posture , This indicates the world coordinate system of the full-motion flight simulator when it returns to its original position. In the world coordinate system of XR headset The lowered position; This represents the pose in the positioning data of a full-motion flight simulator; This indicates the pre-calibrated external parameter position and attitude of the left handle relative to the full-motion flight simulator.

[0092] It should be noted that obtaining the estimated pose of the controller involves converting the positioning information of the full-motion flight simulator to the same world coordinate system as the controller's observation data, which facilitates time comparison and alignment.

[0093] Furthermore, the time deviation between the estimated pose of the left handle and the pose in the fused positioning data of the left handle is calibrated, including:

[0094] ① The estimated pose of the left hand controller obtained in real time is sequentially placed into the first queue, and the fused positioning data of the left hand controller obtained in real time is sequentially placed into the second queue.

[0095] It should be noted that the positioning output frequency of the left and right controllers is higher than that of the full-motion flight simulator. When the length of the first queue is less than the length threshold (e.g., the length threshold is set to 1000), it indicates that the amount of data collected from the full-motion flight simulator is too small. In this case, time deviation calibration is not performed, and the fused positioning data of the left controller output by the controller positioning fusion module is directly transmitted to the full-motion flight simulator positioning module to determine the pose of the full-motion flight simulator in the XR device coordinate system at the initial time. This ensures that the system can still provide basic positioning functions during the initial startup or under specific conditions, maintaining system availability.

[0096] ②When the length of the first queue is greater than or equal to the length threshold, the first queue and the second queue respectively calculate their respective initial angular velocity sequences based on every two adjacent poses.

[0097] The angular velocity in three-dimensional space is calculated based on the attitude (rotation matrix) of two adjacent poses. It is a three-dimensional vector, as shown in the following formula:

[0098] ,

[0099] in, This indicates that based on two adjacent moments in the first queue and Posture in position and The calculated first angular velocity; This indicates that based on two adjacent moments in the second queue and Posture in position and The calculated first angular velocity, This represents the mapping operation from a rotation matrix to a rotation vector.

[0100] Furthermore, the midpoint between the corresponding times of two adjacent poses is taken as the first... The time corresponding to each angular velocity ,Right now Then As an element in the initial sequence of angular velocities for the first queue, As an element in the initial sequence of angular velocities for the second queue.

[0101] ③ Interpolate the initial angular velocity sequences of the first and second queues respectively to obtain optimized angular velocity sequences with timestamp alignment and the same length;

[0102] Specifically, a unified time sampling axis is established, and three scalar component sequences are obtained based on the three spatial components of each angular velocity in the initial angular velocity sequence. Bézier curves are constructed for each scalar component sequence and interpolation calculations are performed. The three interpolated components are then recombined to obtain a dense angular velocity optimization sequence on the unified time axis.

[0103] Through this process, the two angular velocity optimization sequences are unified to the same time base and sampling frequency, with timestamps aligned and of the same length, creating conditions for accurate calibration of subsequent time deviations.

[0104] ④ Taking minimizing the difference between the two optimized angular velocity sequences after time deviation adjustment as the objective function, the objective function is solved by an optimization algorithm to obtain the optimal time deviation.

[0105] It should be noted that the objective function is expressed by the following formula:

[0106] ,

[0107] in, Indicates time deviation; This indicates the length of the angular velocity optimization sequence. This indicates that the optimized angular velocity sequence of the first queue is in angular velocity at time t; This indicates that the angular velocity optimization sequence of the second queue is in angular velocity at time t; The modulus representing the orientation quantity.

[0108] Optimization algorithms, such as those from the Ceres library, are used to solve for the optimal time offset. Time calibration successful.

[0109] Furthermore, the estimated pose of the left handle after time alignment is obtained based on the time deviation, i.e., based on the aligned timestamps. The estimated pose of the left handle is obtained and transmitted to the filtering and fusion module.

[0110] The core task of the filtering and fusion module is to optimally fuse time-aligned positioning data from different sources to obtain a smoother, more accurate, and more reliable optimized pose for the left handle.

[0111] It should be noted that the filtering and fusion module uses an extended Kalman filter to perform fusion filtering based on the uniform motion model, and then extracts the three-dimensional position and three-dimensional rotation vectors from the updated state vector and converts them to obtain the optimized pose of the left handle.

[0112] First, initialize the parameters, including: initialize the state covariance matrix, process noise matrix, observation noise matrix, state transition matrix and observation matrix, and define the state vector.

[0113] Among them, the state covariance matrix and state transition matrix All are initialized to a 9×9 identity matrix; in this embodiment, the time interval is particularly short and the change of the state vector is also very small, so the state transition matrix is ​​approximately an identity matrix.

[0114] Process noise matrix It is a 9×9 diagonal matrix used to describe the measurement noise of the left handle's positioning data; only the diagonal elements are non-zero, and the rest are zero; its diagonal elements correspond to the noise parameters of the left handle's position, rotation (i.e., attitude) and speed, respectively.

[0115] Specifically, the first three elements on the diagonal are the noise of the three-dimensional position of the left handle, which is initialized to 0.001 for example; the middle three elements on the diagonal are the noise of the three-dimensional rotation vector of the left handle, which is initialized to 0.00001 for example; and the last three elements on the diagonal are the noise of the three-dimensional velocity of the left handle, which is initialized to 0.0001 for example.

[0116] Observation noise matrix It is a 6×6 diagonal matrix used to describe the measurement noise of the right handle's positioning data; only the diagonal elements are non-zero, and the rest are zero; its diagonal elements correspond to the noise parameters of the right handle's position and rotation, respectively, and these noise parameters are based on the confidence level of the right handle. The inverse ratio is obtained by setting it.

[0117] Specifically, the first three elements on the diagonal are noise representing the three-dimensional position of the right handle, initialized as follows: The last three elements on the diagonal are noise from the right handle's 3D rotation vector, initialized to... .

[0118] Observation matrix It is a 6×9 matrix. That is, it consists of a 6×6 identity matrix. and a 6×3 zero matrix The system consists of an identity matrix used to obtain the position and rotation vector of the right handle, and a zero matrix used to represent the velocity of the unobserved right handle.

[0119] State vector It is a 9×1 vector. ,in, Indicates three-dimensional position, Represents a three-dimensional rotation vector. Represents three-dimensional velocity. Initial value of the state vector. It is obtained based on the fused positioning data of the left handle when the time calibration is successful, that is .

[0120] Furthermore, in the state prediction stage, the predicted value of the current state vector is obtained based on the state transition matrix and the updated state vector from the previous time step, as shown in the following formula:

[0121] ,

[0122] in, express The predicted state vector value at time t. express The state vector updated at each time step.

[0123] Based on the predefined state transition matrix and process noise matrix, the updated state covariance of the previous time step is predicted to obtain the predicted state covariance value of the current time step, as shown in the following formula:

[0124] ,

[0125] in, and They represent The predicted state covariance at time t and State covariance at time t, This represents the matrix transpose operation.

[0126] During the observation update phase, the observation value at the current moment is dynamically changing, and can be the fused pose in the left handle fused positioning data output by the handle positioning fusion module. It can also be the time-aligned estimated pose of the left handle output by the time alignment module. This design allows the filter to flexibly utilize the most reliable observation source at the current moment: when the handgrip positioning quality is high, the handgrip observation is trusted; when the handgrip may be lost but the full-motion flight simulator data is reliable, the estimated pose calculated by the full-motion flight simulator is trusted.

[0127] from or Extract observation location and observation rotation vector To form the observation vector at the current moment , is a 6×1 vector.

[0128] During the update phase, the observation error at the current moment is calculated based on the observation matrix, the observation vector at the current moment, and the predicted value of the state vector, as shown in the following formula:

[0129] ,

[0130] in, express The observation error at a given time.

[0131] The Kalman gain at the current time is calculated based on the observation matrix, the observation noise matrix, and the predicted state covariance at the current time, as shown in the following formula:

[0132] ,

[0133] in, express Kalman gain at time step.

[0134] Based on the observation error and Kalman gain at the current moment, update the predicted values ​​of the state vector and state covariance at the current moment to obtain the updated state vector and state covariance at the current moment, which are used in the next prediction stage. The formula is shown below:

[0135] ,

[0136] .

[0137] Simultaneously, the 3D position and 3D rotation vectors are extracted from the updated state vector at the current moment and converted into the optimized pose of the left handle at the current moment. This system combines the relative accuracy of the handgrip positioning with the smoothness of the full-motion flight simulator positioning, and performs optimal estimation in the time domain, thus providing the ultimate guarantee for the accuracy and stability of the entire system.

[0138] Finally, the positioning processing system executes the full-motion positioning module to fuse the positioning data of the left hand controller or optimize the pose of the left hand controller with the pre-calibrated extrinsic pose of the full-motion flight simulator. Multiplying by the inverse of the formula yields the pose of the full-motion flight simulator in the XR equipment coordinate system, as shown below:

[0139] ,

[0140] in, This indicates the pose of the full-motion flight simulator in the XR equipment coordinate system. This indicates the fused pose in the left hand controller's fused positioning data, or the optimized pose of the left hand controller. This indicates the pre-calibrated external parameter position and attitude of the left handle relative to the full-motion flight simulator.

[0141] Furthermore, the pose of the full-motion flight simulator in the XR device coordinate system at each moment is sent to the rendering engine of the XR headset in real time. The rendering engine adjusts the trainee's perspective in the virtual world according to this pose, and renders the corresponding virtual scene image in real time and displays it in the XR headset. This achieves synchronization between the visual image and the physical movement of the full-motion flight simulator, greatly enhancing the immersion and realism of the training.

[0142] Compared with existing technologies, this embodiment provides a full-motion flight simulator positioning device based on automatic time deviation calibration, which realizes the linkage between the full-motion flight simulator and the XR headset, and eliminates the need for additional base station positioning equipment, greatly reducing system costs and deployment complexity, and facilitating the large-scale promotion and application of full-motion flight simulators. An automatic time deviation calibration method based on the consistency of rigid body motion angular velocity is proposed, which automatically and accurately calibrates and compensates for the time deviation between the full-motion flight simulator and the XR headset, solving the problem that existing heterogeneous data cannot be directly fused for positioning due to time asynchrony. Through an extended Kalman filter algorithm, time-aligned full-motion flight simulator positioning data and XR controller fused positioning data are deeply fused, complementing each other's advantages. It utilizes both the smoothness of the full-motion flight simulator positioning data and the relative accuracy of the XR controller positioning, effectively avoiding positioning interruptions caused by the failure of a single data source, and ultimately outputting a more stable and accurate pose of the full-motion flight simulator in the XR coordinate system.

[0143] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0144] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A time bias based automatic calibration of a full motion flight simulator positioning device, characterized by, include: An XR headset, a left joystick, and a right joystick; the left and right joysticks are fixed to the full-motion flight simulator, the XR headset is communicatively connected to the full-motion flight simulator, and is equipped with a positioning processing system; the positioning processing system includes: The controller positioning fusion module is used to fuse the positioning data of the left and right controllers to obtain the fused positioning data of the left controller. The time alignment module is used to calculate the estimated pose of the left hand controller based on the positioning data of the full-motion flight simulator, calibrate the time deviation between the estimated pose of the left hand controller and the pose in the fused positioning data of the left hand controller, and obtain the time-aligned estimated pose of the left hand controller based on the time deviation. The calibration of the time deviation between the estimated pose of the left hand controller and the pose in the fused positioning data of the left hand controller includes: sequentially placing the real-time estimated pose of the left hand controller into a first queue, and sequentially placing the real-time fused positioning data of the left hand controller into a second queue; when the length of the first queue is greater than or equal to a length threshold, the first queue and the second queue respectively calculate their respective initial angular velocity sequences based on every two adjacent poses; interpolating the initial angular velocity sequences of the first queue and the second queue respectively to obtain optimized angular velocity sequences with timestamp alignment and the same length; using minimizing the difference between the two optimized angular velocity sequences after time deviation adjustment as the objective function, the objective function is solved through an optimization algorithm to obtain the optimal time deviation. The filtering and fusion module is used to perform fusion filtering on the time-aligned estimated pose of the left hand controller and the fused positioning data of the left hand controller, and output the optimized pose of the left hand controller. The full-motion positioning module is used to locate the pose of the full-motion flight simulator in the XR equipment coordinate system based on the fused positioning data of the left hand handle or the optimized pose of the left hand handle.

2. The time bias based automatic calibration of a full motion flight simulator positioning apparatus of claim 1 wherein, The positioning data includes: pose, confidence level, and velocity; the controller positioning fusion module acquires the fused positioning data of the left controller through the following steps: Based on the relative calibration parameters of the left and right handles, the positioning data of the right handle at the current moment is converted to the left handle to obtain the backup data of the left handle at the current moment. Based on the confidence levels of the left and right controllers at the current moment, obtain the positioning status of the left and right controllers. Based on the comparison of the positioning status of the left and right controllers, the current left controller fused positioning data is obtained according to the backup data of the left controller at the current moment, the positioning data of the left controller at the current moment, or the fused positioning data of the left controller at the previous moment.

3. The time bias based automatic calibration of a full motion flight simulator positioning apparatus of claim 2 wherein, The process of obtaining the current left hand controller fusion positioning data involves obtaining the fusion attitude, fusion position, and fusion speed. The fusion attitude is obtained by selecting the attitude with the larger positioning state from the current left hand controller's backup data and the current left hand controller's positioning data. The fusion position and fusion speed are obtained based on the fusion strategy corresponding to the comparison result.

4. The time bias based automatic calibration of a full motion flight simulator positioning apparatus of claim 1 wherein, In the time alignment module, the estimated pose of the left handgrip is calculated based on the positioning data of the full-motion flight simulator using the following formula: ， wherein, represents the left handle estimated pose, represents the full motion flight simulator world coordinate system under the XR headset world coordinate system at the time of initial full motion flight simulator homing to the original position; represents the pose in the positioning data of the full motion flight simulator; represents the pre-calibrated pose of the left handle relative to the full motion flight simulator.

5. The time bias based automatic calibration of a full motion flight simulator positioning system of claim 1 wherein, The objective function is expressed by the following formula: ， wherein denotes the time offset; denotes the length of the angular velocity optimization sequence, denotes the angular velocity of the angular velocity optimization sequence of the first platoon at time instant; denotes the angular velocity of the angular velocity optimization sequence of the second platoon at time instant; denotes the length of the orientation vector.

6. The time bias based automatic calibration of a full motion flight simulator positioning apparatus of claim 1 wherein, The time alignment module further comprises: when the length of the first queue is less than the length threshold, transmitting the left handle fusion positioning data to a full-motion flight simulator positioning module for positioning the pose of the full-motion flight simulator in the XR device coordinate system.

7. The time bias based automatic calibration of a full motion flight simulator positioning apparatus of claim 1 wherein, The filter fusion module is based on a uniform motion model to perform fusion filtering by using an extended Kalman filter, and then extracts a three-dimensional position and a three-dimensional rotation vector from the updated state vector and converts to obtain a left handle optimized pose; the state vector includes: a three-dimensional position, a three-dimensional rotation vector and a three-dimensional velocity.

8. The time bias based automatic calibration of a full motion flight simulator positioning apparatus of claim 7 wherein, The observation vector in the extended Kalman filter is derived from the left handle fusion positioning data output by the handle positioning fusion module or the time-aligned left handle estimated pose output by the time alignment module.

9. The time bias based automatic calibration of a full motion flight simulator positioning apparatus of claim 1 wherein, The full-motion engine positioning module positions the pose of the full-motion flight simulator in the XR device coordinate system according to the left handle fusion positioning data or the left handle optimized pose by the following formula: ， wherein, represents the pose of the full-motion flight simulator in the XR device coordinate system, represents the fusion pose in the left handle fusion positioning data, or the left handle optimized pose; represents the extrinsic pose of the pre-calibrated left handle relative to the full-motion flight simulator.

Images