Human body pose tracking method and device, robot control method and motion capture glove

Abstract

The invention relates to a human body pose tracking method. The method comprises the following steps: establishing a world coordinate system by using an SLAM (Simultaneous Localization and Mapping) system of head-mounted equipment, and acquiring an absolute pose of the head of a user in the world coordinate system; creating an individualized motion model, and obtaining individualized motion model parameters; acquiring real-time pose data of a distributed IMU node, wherein the distributed IMU node comprises a plurality of IMU units distributed on user limbs; and creating a factor graph optimization model, and determining the human body pose of the user by taking the absolute pose of the head as a root node and combining the individualized motion model parameters and the real-time pose data. The method is high in tracking precision, and can avoid the reduction of the tracking precision of the human body pose caused by integral drift. The invention further provides a human body pose tracking device, a robot control method and motion capture gloves.

Description

Technical Field

[0001] This invention relates to the field of posture determination technology, specifically to a human posture tracking method, device, robot control method, and motion capture glove. Background Technology

[0002] In today's Virtual Reality (VR) and Augmented Reality (AR) applications, precise human pose capture technology plays a crucial role. Human pose capture refers to the precise tracking of the positions of key parts of the human body using specific devices or methods to achieve realistic representations of human movements in virtual or augmented reality.

[0003] Traditional capture methods include optical-based and inertial-based capture systems. Optical-based capture systems typically require a large number of expensive external array cameras to be deployed within the capture area, making them complex to deploy and unusable outdoors or in uncontrolled environments. Inertial-based capture systems rely on multiple inertial measurement units (IMUs) worn on the human body. IMUs rely on accelerometers and gyroscopes for integration. While portable and unobstructed, they suffer from severe integration drift, especially after prolonged operation, where the absolute position of the human body in the world coordinate system diverges, causing the virtual avatar to "slide" or "levitate." Summary of the Invention

[0004] To address the problems of low accuracy and poor anti-drift capability of existing human pose tracking methods, the present invention aims to provide a human pose tracking method with absolute position accuracy and resistance to long-term drift.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A human body pose tracking method includes: establishing a world coordinate system using a head-mounted device's SLAM system and acquiring the absolute pose of the user's head in the world coordinate system; creating an individualized motion model and acquiring individualized motion model parameters; acquiring real-time pose data of distributed IMU nodes, wherein the distributed IMU nodes include multiple IMU units distributed on the user's limbs; and creating a factor graph optimization model, using the absolute pose of the head as the root node, and combining the individualized motion model parameters and the real-time pose data to determine the user's human body pose.

[0006] According to one embodiment of the present invention, the step of creating an individualized motion model and obtaining individualized motion model parameters includes: acquiring three-dimensional scan or multi-view image data of a user, fitting a human body parametric model, optimizing the shape parameters and posture parameters of the human body parametric model to obtain the individualized motion model, and determining individualized motion model parameters based on the individualized motion model.

[0007] According to one embodiment of the present invention, the individualized motion model parameters include bone length data and bone constraint data.

[0008] According to the aforementioned method, the human body pose includes the joint angle vectors of multiple bones of the user and the absolute pose of the user's limb ends in the world coordinate system.

[0009] According to one embodiment of the present invention, the limb end includes fingertips, and the method includes: determining the absolute pose of the fingertips in a world coordinate system based on the absolute pose of the head and joint angle vectors.

[0010] According to one embodiment of the present invention, the head-mounted device includes a head camera, and the method further includes: acquiring a real-time image captured by the head camera; if a visual target is detected in the real-time image, performing image recognition on the visual target in the real-time image to obtain the pose information of the visual target; and correcting the IMU unit according to the pose information of the visual target and the relative positional relationship between the visual target and the IMU unit.

[0011] According to one embodiment of the present invention, after acquiring the real-time image captured by the head camera, the method further includes: if the real-time image does not contain a visual target, then using the IMU unit corrected at the previous moment to determine the human body pose; if the real-time image contains a visual target again, then calculating the residual between the predicted pose of the IMU unit and the observed pose of the head camera, and using the residual update factor graph optimization model to back-correct the IMU unit.

[0012] This invention provides a human posture tracking device, comprising: a head posture determination module, including a head-mounted device with a SLAM system, wherein the SLAM system is used to establish a world coordinate system, and the head posture determination module is used to determine the absolute posture of the user's head in the world coordinate system; an individualized motion model parameter acquisition module, used to acquire individualized motion model parameters; an IMU data acquisition module, used to acquire real-time posture data of distributed IMU nodes; and a posture determination module, used to create a factor graph optimization model, using the absolute posture of the head as the root node, and combining the individualized motion model parameters and the real-time posture data to determine the user's human posture.

[0013] The present invention also provides a method for controlling a robot using the aforementioned method, comprising: acquiring the pose change of a user in the current frame and the previous frame, the pose change including translation increment and rotation increment; scaling the pose change, superimposing the scaled pose change onto the current motion state of the robot, and determining the target pose of the robot at the next moment; the robot performing a corresponding action based on the target pose at the next moment.

[0014] The present invention also provides a motion capture glove for the aforementioned method, wherein a first visual target is provided on the inner side of the wrist of the glove and a second visual target is provided on the outer side of the wrist of the glove. Both the first visual target and the second visual target are provided with IMU units corresponding to them. The relative positions of the first visual target and the second visual target with their corresponding IMU units are determined by pre-calibration.

[0015] According to one embodiment of the present invention, the fingernails of the gloves are further provided with third visual targets, and the relative position of each third visual target and its corresponding IMU unit is determined by pre-calibration.

[0016] The beneficial effects that this invention can achieve are as follows: (1) Drift-free absolute positioning: By anchoring the world coordinates through head SLAM, the problem of divergence of traditional inertial navigation schemes over time is completely solved.

[0017] (2) Omnidirectional anti-occlusion interaction: The multifaceted QR code glove design ensures visual visibility under extreme actions such as clenching fist, turning hand to the side, and flipping wrist, significantly improving the robustness of hand operation.

[0018] (3) The more you use it, the more accurate it becomes: The unique "visual reconnection and correction" mechanism allows users to continuously optimize IMU parameters online during use, rather than the performance deteriorating over time.

[0019] (4) Low cost and high precision: It does not require an optical camera array costing tens of thousands of yuan. It can achieve sub-centimeter level interaction accuracy with just a head-mounted display and low-cost sensors, making it suitable for widespread adoption by consumers. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 For the overall hardware architecture of the system; Figure 2A schematic diagram of a motion capture glove design (showing both sides of the wrist and fingernails); Figure 3 A diagram showing the algorithm and data flow; Figure 4 This is a flowchart of the human pose tracking method in this embodiment; Figure 5 This is a flowchart of the online calibration of the IMU unit in this embodiment. Detailed Implementation

[0021] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0023] See Figure 1 As shown, Figure 1This application provides a schematic diagram of a system for implementing a human pose tracking method. The system includes a head-mounted device (HMU) deployed on the human head, comprising a head camera, a LiDAR, and a head IMU. The IMU in this application is an Inertial Measurement Unit (IMU), which can be a six-axis IMU, including a three-axis accelerometer and a three-axis gyroscope. The six-axis IMU is capable of acquiring real-time 6-DoF pose data. The head camera can be an RGBD camera, capable of simultaneously acquiring color images (RGB) and depth information. Additionally, the system includes distributed IMU nodes, comprising multiple IMU units distributed across the user's limbs. These nodes include the upper arm, forearm, thigh, calf, back of hand, palm, and fingers. The head-mounted device can run a Simultaneous Localization and Mapping (SLAM) algorithm to form a SLAM system. Furthermore, the head camera in the head-mounted device can be used for both visual target detection and limb 3D scanning. LiDAR and head-mounted IMU serve as the hardware devices for the SLAM algorithm, used to output the absolute pose of the head.

[0024] See Figure 2 The image shows the motion capture glove used in the method of this embodiment. This glove is a polyhedral redundant coded glove. A first visual target is provided on the inner side of the glove's wrist (at the flexor carpi radialis tendon), and a second visual target is provided on the outer side of the glove's wrist (at the styloid process of the ulna). This ensures that at least one visual target is within the camera's field of view regardless of whether the user's palm is facing up or the back of their hand is facing up. Additionally, a third visual target is attached to the nail surface of the thumb, index, middle, ring, and little fingers of the glove. When the user clenches their fist, the first visual target may be obscured, but the nail surface usually faces outwards and directly towards the image acquisition unit, ensuring tracking during the "grabbing" action. The visual targets in this embodiment can be any common identifier in the VR, AR, MR, or XR fields. For example, they can be template cards, QR codes, or rune markers drawn with a specific shape. Of course, they can also be custom marking patterns; this embodiment does not limit this. The third visual target can also be a micro QR code (such as a 5mm AprilTag). The relative positions of the first visual target and the second visual target with their corresponding IMU units are determined by pre-calibration, and the relative positions of each third visual target with its corresponding IMU unit are determined by pre-calibration.

[0025] Reference Figure 3The diagram shows the algorithm and data flow of a human pose tracking method provided in one embodiment of this application. The core of this system is a tightly coupled FactorGraph Optimizer based on a sliding window. This optimizer uses the head pose determined by the SLAM system set on the head as the absolute reference, utilizes IMU pre-integration to constrain relative motion, corrects drift through sparse visual observations, and finally calculates the absolute pose of the fingertips using forward kinematics (FK).

[0026] The following will combine Figure 4 The human pose tracking method provided in this embodiment is described in detail. This human pose tracking method includes the following steps: Step S100: Establish a world coordinate system using the SLAM system of the head-mounted device and obtain the absolute pose of the user's head in the world coordinate system; Before using a head-mounted device, factory calibration is required to determine the relative positions of its components. Specifically, factory calibration of the head-mounted camera and head-mounted IMU can be performed to obtain the extrinsic parameter calibration matrix. In this embodiment, after establishing a world coordinate system using a SLAM system, the absolute pose of the user's head in the world coordinate system is obtained. The output frequency of the SLAM system is approximately 30Hz, specifically including the head's position matrix in the world coordinate system. and attitude matrix .

[0027] Step S200: Create an individualized motion model and obtain the parameters of the individualized motion model.

[0028] In this embodiment, personalized kinematic model (PKM) parameters are obtained by establishing a personalized kinematic model. Specifically, a human body parameterized model is fitted using 3D scan or multi-view image data. The human body parameterized model in this embodiment is a general statistical model, including shape parameter β (10-dimensional) and posture parameter θ (72-dimensional), which is a general human body model used to describe the statistical laws of a group, learned from thousands of human body scan data. This includes SMPL (Skinned Multi-Person Linear Model) or SMPL-X (Extended SMPL) parameterized models. By collecting individual data and fitting SMPL / SMPL-X, the shape and posture parameters of the human body parameterized model are optimized to obtain the personalized kinematic model. During the fitting process, optimization methods such as minimizing vertex error can be used. The personalized kinematic model (PKM) in this embodiment is an individual-specific model, retaining both general parameters and individual-specific parameters, optimized for a single individual, and capturing personalized shape / motion features. During the collection of individual data, RGB-D cameras or scanning devices can be used to perform surround scans or multi-view photography of the user, fusing the data to generate a high-precision full-body 3D mesh model, thereby completing the full-body volume reconstruction. Individualized motion parameters include bone length data and bone constraint data. The bone constraint data includes bone parameters such as joint rotation center position, joint rotation axis, and joint range of motion, which serve as rigid geometric constraints in subsequent factor graph optimization.

[0029] Step S300: Obtain real-time pose data of the distributed IMU node, wherein the distributed IMU node includes multiple IMU units distributed on the user's limbs; Distributed IMUs require factory calibration before use, which can be performed on the user in a T-pose configuration. Specifically, the user maintains a T-pose, and the system uses gravity to align the coordinate systems of the IMU units, calculating the rotational offset of each IMU unit relative to the human skeletal coordinate system. R offset .

[0030] In this embodiment, real-time pose data of the distributed IMU nodes is acquired, including the angular velocities of the IMU units located at the upper arm, forearm, back of hand, thigh, and calf. and acceleration The frequency obtained is greater than or equal to 100Hz.

[0031] Step S400: Create a factor graph optimization model, using the absolute pose of the head as the root node, and combine the individualized motion model parameters and the real-time pose data to determine the user's human pose.

[0032] The goal of the human body tracking method in this embodiment is to obtain the user's human body pose, which includes the joint angle vectors of multiple bones of the user and the absolute pose of the user's limbs in the world coordinate system. Specifically, the joint angle vectors include the local rotation angles of multiple bones relative to their parent nodes, while the limb poses include the position and pose of the user's wrists and fingertips in the world coordinate system.

[0033] To ensure real-time computation and rigid constraints, the system state vector, consisting of the variables to be optimized, is constructed as follows:

[0034] in, This represents the head's position, velocity, and rotation angle in the world coordinate system. The acceleration bias and gyroscope bias are zero for all N IMU units in the body; This is a vector of joint angles with M degrees of freedom (DoF) throughout the body (e.g., 3DoF for the shoulder joint, 1DoF for the elbow joint). Instead of directly optimizing the absolute position of each bone, we optimize the joint angles, which naturally satisfies the geometric constraint of constant bone length.

[0035] Using the system's state vector as variables, construct the following nonlinear least squares cost function:

[0036] In the above formula The absolute head pose constraint condition ensures that the root node state of the constraint system remains consistent with the output data of the SLAM system. Its expression is as follows:

[0037] in: The inverse matrix represents the absolute pose matrix (including rotation and translation) of the head in the world coordinate system output by the SLAM system in the head-mounted device (HMU); In this embodiment, the system uses the head position from the optimization variables. and the angle of head rotation Construct the current predicted pose matrix The inverse matrix of the observations of the SLAM system is compared with the predicted values. Multiplying them yields the error between the two, which can then be used to constrain the root node position of the system in factor graph optimization.

[0038] In the formula For IMU pre-integration constraints, for any i-th limb, its pose in the skeletal coordinate system is determined by the head state and joint angles, while the pose of the IMU element needs to be superimposed with the rotational offset of the IMU element relative to the human body's skeletal coordinate system. R offset The pose calculation of the IMU unit is as follows:

[0039] in: The ideal rotation angle of the bones relative to the head is derived from the PKM skeletal chain. This is the static rotation offset matrix of the i-th IMU unit relative to the corresponding skeleton, obtained during the T-pose calibration phase; pass and Construct the following pre-integral residual term :

[0040] The above constraints ensure that the motion state of all IMU units throughout the body is in the head position. and It is derived from the parent coordinate system reference. Therefore, when the SLAM system is set in the head for correction... At that time, the correction amount will be automatically transmitted through the rigid skeletal chain and correct the position of all limbs in the body.

[0041] In the formula For kinematic and skeletal constraints, its expression is as follows:

[0042] in: It is a collection of restricted joints throughout the body. The penalty term based on the barrier function is expressed as follows:

[0043] In the above formula, To optimize the j-th joint angle (such as the elbow flexion-extension angle) among the variables. These are the minimum and maximum allowable rotation angles for the joint, as defined by the PKM individualized model. The weighting factor is the penalty factor for kinematic constraints, and it is usually set to a large value to create a hard constraint effect.

[0044] In the above formula, For visual reprojection constraints, when the head camera is at pixel coordinates Visual target feature points were observed on the hand (or fingernail). Visual reprojection constraints The expression is as follows:

[0045] in: For camera projection model (including distortion parameters); The external parameters of the head camera relative to the head IMU (factory calibration); The pose of the hand in the world frame is derived using the optimized joint angle vectors; This is the extrinsic parameter matrix of the visual target relative to the IMU unit on the back of the hand (factory calibration).

[0046] The constraint-force optimizer adjusts the joint angles of the entire arm (shoulder, elbow, wrist) so that the QR code on the hand is projected exactly onto the pixel seen by the head camera, thereby eliminating cumulative drift.

[0047] In this embodiment, after obtaining the optimized variables through the factor graph optimization model, the absolute pose of the fingertips can also be determined based on the absolute head pose and joint angle vector.

[0048] Specifically, after obtaining the optimized joint angle vectors and the absolute head pose, the absolute position of the fingertips can be calculated using the forward kinematics chain.

[0049] Taking the tip of the right index finger as an example, the calculation path is as follows:

[0050] in: For static rigid body transformation from bone A to bone B (based on bone length in PKM) Decide).

[0051] The local rotation matrix determined by the optimized joint angle θ (e.g., generated by the Rodriguez formula). A fixed transformation from the center of the palm to the fingertips (determined by the hand mesh model in PKM).

[0052] See Figure 5As shown, the method in this embodiment further includes an online zero-bias correction step for the IMU unit, as detailed below: Step S510: If a visual target is detected in the real-time image, image recognition is performed on the visual target in the real-time image to obtain the pose information of the visual target; the IMU unit is corrected according to the pose information of the visual target and the relative position relationship between the visual target and the IMU unit.

[0053] In this embodiment, each visual target corresponds to an IMU unit, and the center points of the visual target and its corresponding IMU unit can be aligned. Their relative positional relationship can be determined through factory calibration, for example... The transformation matrix is ​​determined. By setting a visual target that can be recognized by the head-mounted camera on the palm or fingers, the specific location of the visual target can be determined using image recognition methods. The data observed by the head-mounted camera includes a set of feature points of the visual target. After determining the specific location of the visual target, on the one hand, the position of the finger or palm to which the visual target is attached can be determined, thereby correcting the human body pose. On the other hand, based on the pose information of the visual target and the relative positional relationship between the visual target and the IMU unit, the pose of the IMU unit can be calculated. The pose determined by the visual target can be used to correct the pose value predicted by the IMU unit, thereby correcting the zero bias value of the IMU unit, which includes the angular zero bias. and acceleration zero bias .

[0054] Specifically, a visual reprojection factor with a high weight can be added to the factor graph. By adding the visual reprojection factor, not only can the current hand position be corrected (because the visual target is set on the hand), but the zero bias value of the IMU unit set on the hand can also be updated through graph-optimized backpropagation.

[0055] Step S610: If the real-time image does not include a visual target, the human pose is determined using the IMU unit corrected at the previous moment; if the real-time image does include a visual target again, the residual between the predicted pose of the IMU unit and the observed pose of the head camera is calculated, and the IMU unit is corrected in reverse based on the residual update factor map optimization model.

[0056] Specifically, when all visual targets are lost or occluded, the image recognition method for determining hand position fails. At this point, the pose of the human hand is predicted by integral calculation using the previously corrected IMU unit with precise zero bias and the skeletal constraints determined by PKM. When the visual targets reappear in the field of view and are captured by the head camera, the residual between the predicted pose of the IMU unit and the observed pose of the head camera is calculated. This residual is considered as cumulative drift, and by eliminating this residual, the zero bias of the IMU unit and the human pose are corrected again.

[0057] Specifically, for the fingertips, if a visual target is observed on the fingernail of the glove, the finger joint angle in the above FK chain will be directly corrected by visual constraints; if it is not observed, it will be calculated by the finger joint bending sensor or gesture IMU and superimposed on the absolute pose of the wrist.

[0058] This invention also relates to a method for controlling a robot using the aforementioned human pose tracking method. This method employs an incremental whole-body redirection strategy to control the robot, specifically including the following steps: The system acquires the user's pose change between the current frame and the previous frame, including translation and rotation increments; it scales the pose change and superimposes the scaled pose change onto the robot's current motion state to determine the robot's target pose for the next moment; the robot then performs a corresponding action based on the target pose for the next moment.

[0059] Specifically, it can calculate in real time the pose changes of key parts of the human body (hands, head, torso) between the current frame t and the previous frame t-1. This includes the translation increment Δp and the rotation increment ΔR. The change in the human body's pose is multiplied by a scaling factor. α (Scaling Factor), added to the robot's current motion state, is expressed as follows:

[0060] in: The target pose of the robot; This represents the robot's current pose. This represents the pose superposition operation in Lie algebra space.

[0061] This method allows operators to control robots to cover large workspaces with small hand movements.

[0062] In this embodiment, not only can the pose mapping between the human end and the robot end be realized, but the tilt and rotation changes of the operator's torso can also be mapped into the robot's waist joint movement or chassis movement commands, thereby realizing teleoperation with full-body coordination.

[0063] The embodiments described above are merely examples of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. The scope of protection of this patent should be defined by the appended claims.

Claims

1. A method for human pose tracking, characterized in that, The method includes: A world coordinate system is established using a head-mounted SLAM system, and the absolute pose of the user's head in the world coordinate system is obtained. Create an individualized motion model and obtain the parameters of the individualized motion model; Acquire real-time pose data of distributed IMU nodes, wherein the distributed IMU nodes include multiple IMU units distributed on the user's limbs; A factor graph optimization model is created, with the absolute pose of the head as the root node, and the user's human pose is determined by combining the parameters of the individualized motion model and the real-time pose data.

2. The human pose tracking method according to claim 1, characterized in that, The process of creating an individualized motion model and obtaining individualized motion model parameters includes: The system acquires 3D scans or multi-view image data of the user, fits a human body parametric model, optimizes the shape and posture parameters of the human body parametric model to obtain the individualized motion model, and determines the parameters of the individualized motion model based on the individualized motion model.

3. The human pose tracking method according to claim 2, characterized in that, The individualized motion model parameters include bone length data and bone constraint data.

4. The human pose tracking method according to claim 1, characterized in that, The human body pose includes the joint angle vectors of multiple bones of the user and the absolute pose of the user's limb ends in the world coordinate system.

5. The human pose tracking method according to claim 4, characterized in that, The limb extremities include fingertips, and the method includes: The absolute pose of the fingertip in the world coordinate system is determined based on the absolute head pose and joint angle vector.

6. The human pose tracking method according to claim 1, characterized in that, The head-mounted device includes a head-mounted camera, and the method further includes: Acquire real-time images captured by the head camera; If a visual target is detected in the real-time image, image recognition is performed on the visual target in the real-time image to obtain the pose information of the visual target; the IMU unit is corrected according to the pose information of the visual target and the relative position relationship between the visual target and the IMU unit.

7. The human pose tracking method according to claim 6, characterized in that, After acquiring the real-time image captured by the head camera, the method further includes: if the real-time image does not contain a visual target, then using the IMU unit corrected at the previous moment to determine the human pose; if the real-time image contains a visual target again, then calculating the residual between the predicted pose of the IMU unit and the observed pose of the head camera, updating the factor graph optimization model based on the residual, and back-correcting the IMU unit.

8. A human posture tracking device, characterized in that, The device includes: The head pose determination module includes a head-mounted device with a SLAM system, wherein the SLAM system is used to establish a world coordinate system, and the head pose determination module is used to determine the absolute pose of the user's head in the world coordinate system. The individualized motion model parameter acquisition module is used to acquire individualized motion model parameters. The IMU data acquisition module is used to acquire real-time pose data of distributed IMU nodes; The pose determination module is used to create a factor graph optimization model, which uses the absolute pose of the head as the root node and combines the individualized motion model parameters and the real-time pose data to determine the user's human pose.

9. A method for controlling a robot using the human pose tracking method according to any one of claims 1-7, characterized in that, include: The user's pose change between the current frame and the previous frame is obtained, and the pose change includes translation increment and rotation increment. The pose change is scaled and then superimposed onto the robot's current motion state to determine the robot's target pose at the next moment. as well as The robot performs the corresponding action based on the target pose at the next moment.

10. A motion capture glove for use in any one of the human pose tracking methods according to claims 6-7, characterized in that, The motion capture glove has a first visual target on the inside of the wrist and a second visual target on the outside of the wrist. Both the first and second visual targets are equipped with corresponding IMU units. The relative positions of the first and second visual targets and their corresponding IMU units are determined by pre-calibration.

Images