Automatic calibration method for whole body attitude prediction of sparse IMU (Inertial Measurement Unit)

Abstract

The invention discloses an automatic calibration method for whole-body attitude prediction of a sparse IMU (Inertial Measurement Unit), which comprises the following steps: firstly, after a user wears all measurement modules, swinging a Tpose action towards the true north; the attitude of each measurement module in the NED coordinate system in the process of swinging out the Tpose action is obtained, and the obtained attitude is transmitted to an upper computer PC through a router; a rotation matrix of a human body coordinate system and a geographic coordinate system needing to be calibrated is obtained; storing data; judging a user Tpose calibration action, obtaining an error rotation matrix from a human skeleton to an IMU (Inertial Measurement Unit), and obtaining calibration parameters of an accelerometer and a gyroscope; and correcting the error of the magnetometer by using the UKF until the whole automatic calibration process is finished. According to the method, the calibration action of a user is reduced, the calibration process before action capture of the sparse IMU is simplified, the probability of errors in calibration is reduced, and the reliability of calibration data and the user experience are improved.

Description

Technical Field

[0001] The present invention belongs to the field of human kinematics, and in particular relates to an automatic calibration method for whole-body posture prediction of a sparse IMU. Background Art

[0002] Motion capture technology uses sensors, cameras, or other devices to capture human movements and convert them into digital data. This technology has been widely used in a variety of fields, including film and animation production, virtual reality and augmented reality (VR / AR), sports training and analysis, medicine and rehabilitation, robotics and artificial intelligence, and sports and entertainment games.

[0003] At present, the mainstream motion capture technology is still based on optical motion capture. This technology (such as the Vicon system) requires placing optical markers on the joints of the subject and arranging multiple high-speed cameras (usually using infrared light sources) in the capture area. In order to ensure accurate capture, the optical markers need to be captured by at least three cameras at the same time, which places extremely high demands on the position and number of cameras. A complete motion capture usually takes a lot of time to adjust the position of the camera. In addition, optical motion capture has other defects, such as expensive equipment, sensitivity to lighting conditions and obstructions, complex preliminary preparation processes, and intrusive interference from optical markers, which may limit the natural movement performance of the subject. Due to the limited capture area, the application of optical motion capture in large-scale scenes is also limited, which makes it difficult to apply to consumer-grade application scenarios.

[0004] Another technology is motion capture systems based on inertial measurement units (IMUs), such as Xsens. This type of system captures and records motion data by deploying IMU sensors on the joints of the subject. IMUs are usually composed of accelerometers, gyroscopes, and magnetometers, which can measure acceleration, angular velocity, and magnetic field strength. Although commercial inertial motion capture systems (such as Xsens) can capture human movements more accurately, they still have some shortcomings. For example, the Xsens system requires 17 IMU sensors to be arranged on the human body. Although this dense layout can provide accurate data, it increases the cost of the system and may interfere with the natural movements of the subject.

[0005] To address these issues, researchers have developed motion capture technology based on sparse IMUs. This technology reduces the number of IMUs required while still restoring human posture by combining human kinematics and artificial intelligence algorithms. For example, Sony's Mocopi system uses six IMU sensors to capture the rotation of the head, waist, forearms, and calves, and then infers the rotation of other joints where IMUs are not deployed. However, the Mocopi system fails to completely solve the problem of IMU drift, and therefore requires frequent motion calibration. In addition, the system has poor accuracy in predicting the rotation of joints where IMUs are not deployed, and there is a significant delay in motion restoration, all of which affect the user experience.

[0006] However, the sparse IMU solution requires a rather cumbersome calibration process before use. For example, Mocopi requires six IMUs to be placed on a table (to compensate for gyroscope bias and align their initial heading angles), followed by standing at attention for several seconds as prompted, and finally taking a step forward to complete the calibration. This cumbersome calibration process not only affects the user experience, but if the user miscalibrates a step, the entire motion capture process will be distorted. Summary of the Invention

[0007] Purpose of the invention: In order to overcome the above shortcomings, the purpose of the present invention is to provide an automated calibration method for whole-body posture prediction of a sparse IMU, which reduces the user's calibration movements, simplifies the calibration process of the sparse IMU before motion capture, and at the same time reduces the probability of calibration errors, thereby improving the reliability of the calibration data and the user experience.

[0008] Technical Solution: To achieve the above objectives, the present invention provides an automated calibration method for whole-body posture prediction using a sparse IMU, including the following:

[0009] S1): After the user puts on the measurement module / measuring gloves, they face due north and perform a Tpose action; data is collected and preprocessed;

[0010] S2): AHRS solves and obtains the posture of each measurement unit in the measurement module / measuring glove in the NED coordinate system during the Tpose action, and transmits the posture obtained in the NED coordinate system to the PC host computer through the router. The posture of the measurement unit solved in the i frame in the NED coordinate system is recorded as

[0011] S3): Automatically obtain the rotation matrix of the SMPL coordinate system and the NED geographic coordinate system through the PC host computer

[0012] S4): The PC host computer measures the attitude quaternion of each IMU in the module and the inertial acceleration measured by the IMU Measured angular velocity All are stored in the cache array;

[0013] S5): User Tpose calibration action judgment, if the user Tpose calibration action, the IMU detected

[0014] If the change in the pulling angle exceeds the preset value, it is considered that the automatic calibration has failed;

[0015] S6): When the user performs the Tpose calibration action, if the change amplitude of the Euler angle detected in the IMU is less than the preset value and the set calibration time has not been reached, then return to S4); if the change amplitude of the Euler angle detected in the IMU is less than the preset value and the set calibration time has been reached, then calculate the average attitude quaternion of the IMU from the cache array, and the average attitude quaternion of the IMU is converted into a rotation matrix after normalization;

[0016] S7): Obtain calibration parameters of the accelerometer and gyroscope;

[0017] S8): Use the palm-joining motion to calibrate the error matrix of the finger skeleton and the measurement unit deployed on the finger skeleton.

[0018] The automated calibration method for whole-body posture prediction of the sparse IMU described in the present invention, said S2)

[0019] The AHRS solver obtains the posture of each measurement unit in the measurement module / glove in the NED coordinate system during the Tpose action. A distributed MCU is used to calculate the posture of each measurement unit in the measurement module / glove in the NED coordinate system and calibrate the error of the IMU relative to the human skeleton. The specific calibration process is as follows:

[0020] The IMU's deployment position may be misaligned with the corresponding bones due to muscle, skin, or wearing method. Usually, the error of the IMU relative to the human skeleton is calibrated using Tpose / Apose / Standing Attention posture, as follows:

[0021]

[0022] in, The i-th frame pose of the skeleton of the deployed measurement unit is represented in the SMPL coordinate system, as described in S3), has the properties of an orthogonal matrix, so is the error matrix from the human skeleton to the corresponding IMU.

[0023] The automatic calibration method for whole-body posture prediction of sparse IMU described in the present invention calibrates the error of IMU relative to human skeleton. Both are identity matrices I 3×3 ,Right now

[0024] in, It is the effective rotation matrix of the IMU body coordinate system in the geographic coordinate system during the Tpose action, t0 is the start time of the Tpose action, and t1 is the end time of the Tpose action;

[0025] It is the error matrix from human skeleton to IMU, which is a unit orthogonal matrix. The specific solution process is as follows:

[0026]

[0027] The automatic calibration method for whole body posture prediction of sparse IMU described in the present invention, in said S3) in obtaining the rotation matrix After the calibration, the IMU attitude in the geographic coordinate system needs to be converted to the SMPL coordinate system. The specific calibration process is as follows:

[0028] The xyz axes of the human body coordinate system (SMPL coordinate system) correspond to the sagittal plane normal vector in the left direction, the transverse plane normal vector in the upward direction, and the coronal plane normal vector in the forward direction, which has a rotation deviation from the North East Earth (NED) geographic coordinate system;

[0029] The specific deviations are as follows:

[0030]

[0031] Among them, v e is the representation of any vector in the geographic coordinate system, v s is the representation of any vector in the SMPL coordinate system.

[0032] The automated calibration method for whole-body posture prediction of a sparse IMU described in the present invention obtains calibration parameters of the accelerometer and gyroscope in S7), that is, calculates the acceleration and angular velocity errors from the cache data group in S4), and calibrates the IMU and magnetometer using an error model consisting of inertial acceleration error, angular velocity error, and magnetometer error. The error model is as follows:

[0033]

[0034] in, is the true inertial acceleration, is the inertial acceleration measured by IMU, Ta is the small rotation matrix for accelerometer correction, K a is the accelerometer scale factor, b′ a is the accelerometer bias, ε a is the accelerometer white noise, assuming ε a is a zero mean value with variance σ a Gaussian process;

[0035] is the true angular velocity, is the angular velocity measured by IMU, T ω is the small rotation matrix of the gyroscope correction, K ω is the gyroscope scale factor, b′ ω is the gyroscope bias, ε ω is the gyroscope white noise.

[0036] The automated calibration method for whole-body posture prediction of a sparse IMU of the present invention, in the user Tpose calibration action in S6), it is assumed that all IMUs are relatively stationary during the Tpose action, so the least squares method is used to fit the zero bias, scale factor, and small rotation matrix of the IMU, and then the corrected IMU reading is obtained by correction, and the corrected acceleration and corrected angular velocity are obtained, and the error parameters of the IMU acceleration and IMU angular velocity are calculated:

[0037] The objective function of IMU acceleration is:

[0038]

[0039] in, is the value of the real inertial acceleration of the i-th frame, and N is the length of the cache array;

[0040] g is the local acceleration, which is obtained by minimizing J a Fit T a , K a , b′ a , the T a , K a , b′ a

[0041] They are the small rotation matrix, accelerometer scale factor, and accelerometer bias for accelerometer correction respectively;

[0042] The objective function of the IMU angular velocity is

[0043] in, is the value of the true angular velocity of the i-th frame,

[0044] By minimizing J ω Fit Tω , K ω , b′ ω To obtain more accurate motion acceleration, these calibration parameters are then sent to each MCU via the host PC through the router to obtain more accurate inertial information;

[0045] Among them, T ω , K ω , b′ ω They are the tiny rotation matrix of gyroscope correction, gyroscope scale factor, and gyroscope zero bias respectively.

[0046] The automated calibration method for whole-body posture prediction of a sparse IMU of the present invention, wherein in S6), the average posture quaternion of the IMU is calculated in the cache array, and the specific process of converting the average posture quaternion into a rotation matrix after normalization is as follows:

[0047]

[0048] in, The i-th frame pose quaternion of the IMU deployed at the specified joint position in the cache array. The pose quaternion is the unit pose quaternion rotated from the geographic coordinate system to the IMU body coordinate system;

[0049] The average attitude quaternion of the IMU deployed at the specified joint position in the cache array;

[0050] is the normalized average attitude quaternion, its real part is q0, and its three imaginary parts are q1, q2, and q3 respectively;

[0051] is the effective rotation matrix of the IMU body coordinate system in the geographic coordinate system during the T action, that is,

[0052] In the automated calibration method for whole-body posture prediction of a sparse IMU of the present invention, UKF is used to correct the magnetometer error in S1), and the magnetometer correction is performed in real time by the UKF algorithm in the MCU.

[0053] The magnetometer measurements can be modeled as:

[0054] B k =(I+D) -1 (A k H k +b+ε k ), k = 1, ..., N

[0055] Among them, B k is the measurement value of the magnetic field by the magnetometer at time k, H kis the corresponding value of the geomagnetic field related to the Earth's fixed coordinate system, A k is the unknown attitude matrix of the magnetometer relative to the geographic coordinate system, I is the 3×3 identity matrix, D is the matrix of the magnetometer's scale factor error and non-orthogonal row error (inter-axis coupling), b is the magnetometer zero bias vector, ε k is the measurement noise vector, assuming that the measurement noise vector is a vector with zero mean and covariance Σ k Gaussian process;

[0056] z k The measurement equation is:

[0057] z k =||B k ||2-||H k || 2

[0058] Expand and organize k Measurement equation:

[0059]

[0060] In which, define v k is the error term of the noise part:

[0061] v k =2((I+D)B k -b) T ε k -||ε k || 2

[0062] v k Approximately Gaussian noise, so its mean μ k and variance for:

[0063] μ k =E{v k =E{2((+D)B k -b) T ε k -||ε k || 2}=-Tr(∑ k )

[0064]

[0065] Among them, E{} is the expectation of the variable, Tr() is the trace of the variable, σ k is a function of the magnetometer's scale factor error and non-orthogonal row error matrix D and the magnetometer's zero bias vector b. To estimate D and b, define the following variables:

[0066] F=2D+D 2

[0067] f=[F 11 ,F 22 ,F 33 ,F 12 ,F 13 ,F 23 ] T

[0068] c=(I+D)b

[0069]

[0070] Among them, F is a symmetric matrix, F ij Represents the element in the i-th row and j-th column of the symmetric matrix F; ij = 11, 22, 33, 12, 13, 23;

[0071] f is a column vector with 6 rows and 1 column, consisting of the elements of the symmetric matrix F;

[0072] c is a column vector with 3 rows and 1 column, which is calculated from I, D, and b;

[0073] B i，k Indicates B k The i-th row vector of the matrix, which is a vector with 1 row and 3 columns; Indicates B k The matrix of the product of the i-th row vector of the matrix, which is a matrix with 3 rows and 3 columns; Indicates B k The matrix is the product of the i-th row vector and the j-th row vector. It is a 3-row, 3-column matrix. The permutations of i and j are i=1 and j=2, i=1 and j=3, and i=2 and j=3.

[0074] L k Indicated by and -S k The matrix formed by splicing;

[0075] θ is a 9-row, 1-column column vector consisting of c and f;

[0076] So reorganize z k have:

[0077]

[0078] Wherein, b(c, f) is the function of b represented by c and f, and b(θ) is the function of b represented by θ;

[0079] Then, as long as the symmetric matrices F and c are determined, D and b can be derived. The specific derivation process is as follows:

[0080] Perform eigendecomposition on the symmetric matrix F as follows:

[0081] F=UVU T

[0082] Among them, U is an orthogonal matrix, that is, the eigenvector matrix of F;

[0083] V=diag(V 11 ,V 22 ,V 33 ) is a diagonal matrix, and the diagonal elements are the eigenvalues of F;

[0084] Eigenvector definition Fu i as follows:

[0085] Fu i =UVU T u i =Udiag(0,…,V ii ,…,0)=V ii u i

[0086] Among them, u i is the i-th column vector of the orthogonal matrix U, V ii is the i-th diagonal element of the matrix V; then define a new diagonal matrix W = diag(W 11 ,W 22 ,W 33 ), whose diagonal elements are obtained by the following formula:

[0087] where j = 1, 2, 3;

[0088] So D and b are obtained as follows:

[0089] D=UWU T

[0090] b=(I+D) -1 c

[0091] Then define θ of the kth frame, that is, θ k is the system status; z k For the measurement equation, we can construct the following Carl

[0092] Mann filter equation:

[0093]

[0094] The variables are then iteratively optimized using UKF, and the prediction and correction steps follow the standard unscented Kalman filter method;

[0095] At this point, the values of D and b can be estimated in real time to obtain more accurate magnetometer readings.

[0096] During the calibration process, if the predicted whole-body posture includes fingers, the palm-joining calibration in S8) is performed to optimize the error matrix from the finger bones to the corresponding IMU. Since the five fingers of the palm are straightened in the T movement, the angles between adjacent fingers vary from 3 to 45 degrees, making it difficult for the user to perform a standard finger movement. When the five fingers are straightened, the other four fingers except the thumb are parallel, and the thumb and index finger form a 45-degree angle, the palm-joining movement is relatively easy for the user to perform, so a more accurate calibration is performed using the palm-joining movement.

[0097] Since the Tpose motion calibration was performed before, the posture of the palm-dorsal bones in the glove measured in the i-th frame in the SMPL coordinate system is

[0098]

[0099] in, The posture of the IMU deployed on the back of the palm in the geographic coordinate system at the i-th frame,

[0100] The error matrix from the palm-dorsal skeleton to the IMU deployed on the palm-dorsal;

[0101] is calculated as follows:

[0102]

[0103] in, The effective rotation matrix of the IMU deployed on the back of the palm in the geographic coordinate system during the T maneuver.

[0104] The automated calibration method for whole-body posture prediction of the sparse IMU described in the present invention, wherein the specific process of calibrating the finger skeleton and the error matrix of the measurement unit deployed on the finger skeleton using the palm-joining action in S8) is as follows:

[0105] The palm-jointing calibration is similar to the Tpose action calibration method. During calibration, the attitude quaternion of each measurement unit in the measuring glove is stored in the cache array, and the Euler angle is used to detect whether there is a large jitter in the calibration action. After the calibration action is completed, the attitude quaternion is taken from the cache array and the effective rotation matrix of the IMU body coordinate system in the geographic coordinate system during the palm-jointing calibration is calculated.

[0106] Among them, g0 is the start time of the palm-joining action, and g1 is the end time of the palm-joining action. The solution process is the same as that of the Tpose action calibration;

[0107] It is known that when the SMPLH model is in the palm-to-palm action, the rotation matrix A of the finger bones relative to the palm back bones is 3×3 ;

[0108] During the Tpose action, the spatial position of the middle finger proximal phalanx joint node in the SMPL coordinate system is obtained according to the SMPLH model: Get the spatial position of the distal phalanx joint node of the middle finger

[0109] So we can find the vector of the middle finger:

[0110]

[0111] Similarly, the vectors V of the other four fingers can be obtained thumb , V index , V ring , V pinky ;

[0112] The thumb vector and index finger vector of the left hand rotate a specific angle θ around the positive direction of the y-axis of the SMPL coordinate system t ,0°<θ t <90°、θ i , 0°<θ i <90°, the left hand's ring finger and little finger are rotated around the negative direction of the y-axis of the SMPL coordinate system by a specific angle θ r , 0°<θ r <90°、θ p , 0°<θ p The included angle between the projection of the index finger vector of the left hand and the projection of the middle finger vector on the xoz plane of the SMPL coordinate system is 45°. The included angle between the projection of the index finger vector, the ring finger vector and the little finger vector on the xoz plane of the SMPL coordinate system and the projection of the middle finger vector on the xoz plane of the SMPL coordinate system is 0°.

[0113] When the palms are together, the rotation matrix of the left thumb phalanx relative to the left palm back bone is:

[0114]

[0115] Similarly, the rotation matrix A of the left index finger phalanges, ring finger phalanges, and little finger phalanges relative to the left palm back bones can be obtained: li 、A lr 、A lp , then the rotation matrix of the left middle finger phalanx relative to the left palm back bone is the identity matrix A lm =I 3×3 ;

[0116] Then, the relative rotation matrix of the phalanges of the right hand relative to the right palm back bones is obtained through mirroring;

[0117] Therefore, the palm-joining action is used to more accurately obtain the error matrix from the bones of the ten fingers to the IMU:

[0118]

[0119] in, The effective rotation matrix of the IMU deployed on the dorsal palm bone in the geographic coordinate system during the palm-closing action;

[0120] in, The effective rotation matrix of the IMU deployed on the finger phalanges in the geographic coordinate system during the palm-joining action;

[0121] in, This is the error matrix from the finger bones to the IMU to be further optimized;

[0122] then This completes the palm-joint calibration.

[0123] It can be seen from the above technical solution that the present invention has the following beneficial effects:

[0124] 1. The automated calibration method for whole-body posture prediction of the sparse IMU described in the present invention reduces the user's calibration movements, simplifies the calibration process before the sparse IMU performs motion capture, and reduces the probability of calibration errors, thereby improving the reliability of the calibration data and the user experience.

[0125] 2. The automated calibration method for whole-body posture prediction of the sparse IMU described in the present invention can automatically complete the calibration of the above-mentioned IMU, coordinate system rotation matrix, and bone error within a few seconds when the user poses Tpose, greatly shortening the calibration time, effectively improving its detection efficiency, and effectively ensuring the accuracy of the calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0126] Figure 1 Flowchart of the automated calibration method for whole-body posture prediction of the sparse IMU according to the present invention;

[0127] Figure 2 This is a schematic diagram of the deployment of the measurement module and the measurement glove in the present invention;

[0128] Figure 3 Schematic diagram of the palm-joining gesture in the present invention;

[0129] Figure 4 This is a schematic diagram of the electrical connections of the measurement module in the present invention;

[0130] Figure 5 This is a schematic diagram of router networking in the present invention;

[0131] Figure 6 This is a flow chart of the data synchronization algorithm in the present invention. DETAILED DESCRIPTION

[0132] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0133] Example 1

[0134] like Figure 1 The automated calibration method for whole-body posture prediction of a sparse IMU is shown, including: the specific calibration method is as follows:

[0135] S1): First, the user puts on all measurement modules / gloves and strikes a Tpose facing true north. After the motion capture system or device is powered on, data is collected. The distributed MCU uses the UKF to observe the magnetometer's zero bias, scale factor, and non-orthogonality error in real time. The raw magnetometer and IMU readings are then preprocessed.

[0136] S2): AHRS solves and obtains the posture of each measurement unit in the measurement module / measuring glove in the NED coordinate system during the Tpose action. That is, the distributed MCU calculates the posture of each measurement unit in the measurement module / measuring glove in the NED coordinate system, and then transmits the posture of each measurement unit in the NED coordinate system to the PC host computer through the router. The posture of the measurement unit solved in the NED coordinate system of the i frame is recorded as

[0137] S3): When the user performs the T-pose facing due north, the user is on the horizontal ground facing due north, and the relative orientation of the left upper front and geographic north and east axes of the human body is fixed. By automatically obtaining the rotation matrix of the SMPL coordinate system and the NED geographic coordinate system for And it has the properties of an orthogonal matrix;

[0138] S4): Data storage, that is, the host PC stores the attitude quaternion of each IMU and the inertial acceleration measured by the sensor Measuring angular velocity All are stored in the cache array;

[0139] S5): User Tpose calibration action judgment. If the change amplitude of the Euler angle detected in the IMU exceeds the preset value (such as 10°) during the user Tpose calibration action, it means that the user Tpose calibration action

[0140] If there is a large jitter, it is considered that the automatic calibration has failed;

[0141] S6): When the user performs the Tpose calibration action, if the change amplitude of the Euler angle detected in the IMU is less than the preset value and the set calibration time has not been reached, return to S4); if the change amplitude of the Euler angle detected in the IMU is less than the preset value and the set calibration time (for example, 5 seconds) has been reached, the average attitude quaternion, acceleration and angular velocity are calculated from the cache array, and the average attitude quaternion is converted into a rotation matrix after normalization;

[0142] S7): Obtain calibration parameters of the accelerometer and gyroscope; that is, calculate the errors of acceleration and angular velocity from the cache data group in S4);

[0143] S8): The finger bones and the error matrix of the measurement unit deployed on the finger bones are calibrated using the palm-joining action. The entire automated calibration process is now complete.

[0144] The specific process of data preprocessing of the raw magnetometer readings and raw IMU readings obtained in S1) is as follows:

[0145] The original magnetometer readings are corrected to obtain a corrected magnetometer; the original IMU readings are fitted with the IMU zero bias, scale factor, and small rotation matrix using the least squares method by the microcontroller unit in each measurement module / glove, and then the corrected IMU readings are obtained through correction, namely the corrected acceleration and corrected angular velocity;

[0146] The corrected data are then fused, and the corrected magnetometer, corrected acceleration, and corrected angular velocity are fused through the AHRS system to obtain the rotational posture and geographic acceleration of the joints corresponding to the deployment position of each measurement module, as well as the rotational posture and geographic acceleration of the palm back, proximal phalanx of the thumb, index finger, middle finger, ring finger, and middle phalanx of the little finger of each measurement glove.

[0147] It should be noted that the difference between the maximum and minimum Euler angles in the user's Tpose action is less than 10°.

[0148] In this embodiment, the specific process of the distributed MCU in S2) respectively calculating the posture of each measurement unit in the measurement module / measuring glove in the NED coordinate system is as follows: the north-east coordinate system is E, and the IMU coordinate system is S;

[0149] The data returned by the IMU's three-axis gyroscope is the angular velocity of the system around the x-axis, y-axis, and z-axis. The three angular velocities are expressed as ω. x 、ω y 、ω z To express it, the data returned by the gyroscope can be regarded as an attitude quaternion with a real part of zero, and the corrected angular velocity ω is used. S To express:

[0150]

[0151] Attitude quaternion change speed and the current attitude quaternion and angular velocity ω S The details are as follows:

[0152]

[0153] The attitude quaternion at time t-1 is known and angular velocity ω S t-1 , and the angular velocity ω at time t S t , the system sampling interval is Δt, find the attitude quaternion at time t

[0154]

[0155] Among them, K1 is the preliminary estimate of the attitude quaternion change, which represents the attitude quaternion change rate caused by the gyroscope angular velocity information;

[0156] K2 is the attitude quaternion update calculated by adding the attitude quaternion at time t-1 to the time step correction;

[0157] The average change rate of the object posture quaternion between time t-1 and time t is approximately estimated to be In practice, we often cannot get the exact attitude quaternion at time t-1, but an optimal estimate. So the attitude quaternion from the north-east coordinate system to the sensor coordinate system updated based on the gyroscope data at time t is as follows:

[0158]

[0159] Assuming that the gyroscope calculates the attitude accurately, then the E-frame gravity vector The acceleration vector g of the attitude solution in the S system S t Should be:

[0160]

[0161] Acceleration vector g for attitude solution S t and the actual accelerometer measured The deviation is e S a,t =cross(a S t , g St )

[0162] Assume that the corrected magnetometer measurement value in the S system is After rotating to E system:

[0163]

[0164] Since the E system is a north-east coordinate system, the geomagnetic field lines only have components on the x-axis and z-axis, so the theoretical output value of the magnetometer in the E system is b E t for:

[0165]

[0166] Theoretically y ≈0, but there is an error h in the actual posture solution y ≠0, so:

[0167]

[0168] The theoretical magnetometer output value b of the E system E t Then rotate to S system and get W S t :

[0169]

[0170] Then calculate W S t and the corrected magnetometer measurement value m in the S system S t Deviation:

[0171] e S m,t =cross(m S t,W S t )

[0172] So the total error is:

[0173] e S t =e S a,t +e S m,t

[0174] The gyroscope is compensated for deviation through a PI controller:

[0175]

[0176] The compensated attitude quaternion is then updated as follows:

[0177]

[0178] Normalized attitude quaternion, final attitude update:

[0179]

[0180] In the automated calibration method for whole-body posture prediction of the sparse IMU described in this embodiment, the distributed MCU in S2) calculates the posture of each measurement module in the NED coordinate system, and the error of the IMU relative to the human skeleton needs to be calibrated. The specific calibration process is as follows:

[0181] The IMU's deployment position may be misaligned with the corresponding bones due to muscle, skin, or wearing method. Usually, the error of the IMU relative to the human skeleton is calibrated using Tpose / Apose / Standing Attention posture, as follows:

[0182]

[0183] in, The i-th frame pose of the skeleton of the deployed measurement unit is represented in the SMPL coordinate system, as described in S3), has the properties of an orthogonal matrix, so is the error matrix from the human skeleton to the corresponding IMU.

[0184] The automatic calibration method for whole-body posture prediction of sparse IMU described in this embodiment is to calibrate the error of IMU relative to human skeleton. Both are identity matrices I 3×3 ,Right now

[0185] in, It is the effective rotation matrix of the IMU body coordinate system in the geographic coordinate system during the Tpose action, t0 is the start time of the Tpose action, and t1 is the end time of the Tpose action;

[0186] It is the error matrix from human skeleton to IMU, which is a unit orthogonal matrix. The specific solution process is as follows:

[0187]

[0188] The automatic calibration method for whole body posture prediction of sparse IMU described in this embodiment, in S3) obtaining the rotation matrix After the calibration, the IMU attitude in the geographic coordinate system needs to be converted to the SMPL coordinate system. The specific calibration process is as follows:

[0189] The xyz axes of the SMPL coordinate system (human body coordinate system) correspond to the sagittal plane normal vector in the left direction, the transverse plane normal vector in the upward direction, and the coronal plane normal vector in the forward direction, which has a rotation deviation from the North East Earth (NED) geographic coordinate system;

[0190] The specific deviations are as follows:

[0191]

[0192] Among them, v e is the representation of any vector in the geographic coordinate system, v s is the representation of any vector in the SMPL coordinate system.

[0193] In the automated calibration method for whole-body posture prediction of the sparse IMU described in this embodiment, due to manufacturing errors, there is a deviation between the output acceleration and angular velocity of the IMU and the actual value. For example, in a stationary state, the gyroscope should output a very small reading (containing only the angular velocity reading of the Earth's rotation, 7.2722×10^-5rads / s), but the actual output has a deviation, and the output value may reach 0.05rads / s. This zero bias can cause significant errors during long-term integration, resulting in posture drift.

[0194] In S7), the calibration parameters of the accelerometer and gyroscope are obtained, that is, the errors of acceleration and angular velocity are calculated from the cache data group of S4), that is, the error model composed of inertial acceleration error, angular velocity error and magnetometer error is used to calibrate the IMU and magnetometer. The error model is as follows:

[0195]

[0196] in, is the true inertial acceleration, is the inertial acceleration measured by IMU, T a is the small rotation matrix for accelerometer correction, K a is the accelerometer scale factor, b′ a is the accelerometer bias, ε a is the accelerometer white noise, assuming ε a is a zero mean value with variance σ a Gaussian process;

[0197] is the true angular velocity, is the angular velocity measured by IMU, T ω is the small rotation matrix of the gyroscope correction, K ω is the gyroscope scale factor, b′ ω is the gyroscope bias, ε ωis the gyroscope white noise.

[0198] In the automated calibration method for whole-body posture prediction of the sparse IMU described in this embodiment, during the user's Tpose calibration action in S6), since it is assumed that all IMUs are relatively stationary during the Tpose action, the least squares method is used to fit the zero bias, scale factor, and small rotation matrix of the IMU. Then, the corrected IMU reading is obtained by correction, and the corrected acceleration and corrected angular velocity are obtained. The error parameters of the IMU acceleration and IMU angular velocity are calculated:

[0199] The objective function of IMU acceleration is:

[0200]

[0201] in, is the value of the real inertial acceleration of the i-th frame, and N is the length of the cache array;

[0202] g is the local acceleration, which is obtained by minimizing J a Fit T a , K a , b′ a , the T a , K a , b′ a

[0203] They are the small rotation matrix, accelerometer scale factor, and accelerometer bias for accelerometer correction respectively;

[0204] The objective function of the IMU angular velocity is

[0205] in, is the value of the true angular velocity of the i-th frame,

[0206] By minimizing J ω Fit T ω , K ω , b′ ω To obtain more accurate motion acceleration, these calibration parameters are then sent to each MCU via the host PC through the router to obtain more accurate inertial information;

[0207] Among them, T ω , K ω , b′ ω They are the tiny rotation matrix of gyroscope correction, gyroscope scale factor, and gyroscope zero bias respectively.

[0208] In the automated calibration method for whole-body posture prediction of the sparse IMU described in this embodiment, the specific process of calculating the average posture quaternion of the IMU in the cache array in S6) and converting the average posture quaternion into a rotation matrix after normalization is as follows:

[0209]

[0210] in, The i-th frame pose quaternion of the IMU deployed at the specified bone position in the cache array. The pose quaternion is the unit pose quaternion rotated from the geographic coordinate system to the IMU body coordinate system;

[0211] The average pose quaternion of the IMU deployed at the specified bone position in the cache array;

[0212] is the normalized average attitude quaternion, its real part is q0, and its three imaginary parts are q1, q2, and q3 respectively;

[0213] is the effective rotation matrix of the IMU body coordinate system in the geographic coordinate system during the T action, that is,

[0214] In the automated calibration method for whole-body posture prediction of a sparse IMU described in this embodiment, the UKF is used in S1) to observe the zero bias, scale factor, and non-orthogonal error of the magnetometer in real time; the original magnetometer reading is corrected to obtain a corrected magnetometer, and the magnetometer error is corrected using the UKF. The magnetometer correction is performed in real time by the UKF algorithm in the MCU. The magnetometer measurement can be modeled as follows:

[0215] B k =(I+D) -1 (A k H k +b+ε k ),k=1,...,N

[0216] Among them, B k is the measurement value of the magnetic field by the magnetometer at time k, H k is the corresponding value of the geomagnetic field related to the Earth's fixed coordinate system, A k is the unknown attitude matrix of the magnetometer relative to the geographic coordinate system, I is the 3×3 identity matrix, D is the matrix of the magnetometer's scale factor error and non-orthogonal row error (inter-axis coupling), b is the magnetometer zero bias vector, ε k is the measurement noise vector, assuming that the measurement noise vector is a vector with zero mean and covariance Σ k Gaussian process;

[0217] zk The measurement equation is:

[0218] z k =||B k || 2 -||H k || 2

[0219] Expand and organize k Measurement equation:

[0220]

[0221] In which, define v k is the error term of the noise part:

[0222] v k =2((I+D)B k -b) T ε k -||ε k || 2

[0223] v k Approximately Gaussian noise, so its mean μ k and variance for:

[0224] μ k =E{v k =E{2((+D)B k -b) T ε k -||ε k || 2}=-Tr(∑ k )

[0225]

[0226] Among them, E{} is the expectation of the variable, Tr() is the trace of the variable, σ k is a function of the magnetometer's scale factor error and non-orthogonal row error matrix D and the magnetometer's zero bias vector b. To estimate D and b, define the following variables:

[0227] F=2D+D 2

[0228] f=[F 11 ,F 22 ,F 33 ,F 12 ,F 13 ,F 23 ] T

[0229] c=(I+D)b

[0230]

[0231] Among them, F is a symmetric matrix, F ij Represents the element in the i-th row and j-th column of the symmetric matrix F; ij = 11, 22, 33, 12, 13, 23;

[0232] f is a column vector with 6 rows and 1 column, consisting of the elements of the symmetric matrix F;

[0233] c is a column vector with 3 rows and 1 column, which is calculated from I, D, and b;

[0234] B i,k Indicates B k The i-th row vector of the matrix, which is a vector with 1 row and 3 columns; Indicates B k The matrix of the product of the i-th row vector of the matrix, which is a matrix with 3 rows and 3 columns; Indicates B k The matrix is the product of the i-th row vector and the j-th row vector. It is a 3-row, 3-column matrix. The permutations of i and j are i=1 and j=2, i=1 and j=3, and i=2 and j=3.

[0235] L k Indicated by and -S k The matrix formed by splicing;

[0236] θ is a 9-row, 1-column column vector consisting of c and f;

[0237] So reorganize z k have:

[0238]

[0239] Among them, b(c,f) is the function of b represented by c and f, and b(θ) is the function of b represented by θ;

[0240] Then, as long as the symmetric matrices F and c are determined, D and b can be derived. The specific derivation process is as follows:

[0241] Perform eigendecomposition on the symmetric matrix F as follows:

[0242] F=UVU T

[0243] Among them, U is an orthogonal matrix, that is, the eigenvector matrix of F;

[0244] V=diag(V 11 ,V 22 ,V 33) is a diagonal matrix, and the diagonal elements are the eigenvalues of F;

[0245] Eigenvector definition Fu i as follows:

[0246] Fu i =UVU T u i =Udiag(0,…,V ii …,0)=V ii u i

[0247] Among them, u i is the i-th column vector of the orthogonal matrix U, V ii is the i-th diagonal element of the matrix V; then define a new diagonal matrix W = diag(W 11 ,W 22 ,W 33 ), whose diagonal elements are obtained by the following formula:

[0248] where j = 1, 2, 3;

[0249] So D and b are obtained as follows:

[0250] D=UWU T

[0251] b=(I+D) -1 c

[0252] Then define θ of the kth frame, that is, θ k is the system status; z k For the measurement equation, we can construct the following Carl

[0253] Mann filter equation:

[0254]

[0255] The variables are then iteratively optimized using UKF, and the prediction and correction steps follow the standard unscented Kalman filter method;

[0256] At this point, the values of D and b can be estimated in real time to obtain more accurate magnetometer readings.

[0257] Since the system is a discrete system, θ k The K in the formula represents the variable value at time K. UKF derives the variable value at the next discrete time k+1 based on the previous discrete time k. The system state is the state variable in modern control theory, which is the minimum set of variables that is sufficient to completely determine the operating state of the system.

[0258] It should be noted that UKF optimizes theta so that each step of observation zk The covariance of the residuals between the predictions of the prediction model and the LSTM neural network is minimized.

[0259] During the calibration process, if the whole body posture prediction includes the finger part, the palm calibration in S8) is performed to optimize the error matrix from the finger bones to the corresponding IMU. Since the five fingers of the palm are straight in the T action, the angles between adjacent fingers vary from 3 to 45 degrees, making it difficult for the user to make standard finger movements; however, if Figure 2 , five fingers are straightened, the other four fingers except the thumb are parallel, and the thumb and index finger are at a 45° angle. The palm-to-palm action is relatively easy for users to do, so a more accurate calibration is achieved by using the palm-to-palm action;

[0260] Since the Tpose motion calibration was performed before, the posture of the palm-dorsal bones in the glove measured in the i-th frame in the SMPL coordinate system is

[0261]

[0262] in, The posture of the IMU deployed on the back of the palm in the geographic coordinate system at the i-th frame,

[0263] The error matrix from the palm-dorsal skeleton to the IMU deployed on the palm-dorsal;

[0264] is calculated as follows:

[0265]

[0266] in, The effective rotation matrix of the IMU deployed on the back of the palm in the geographic coordinate system during the T maneuver.

[0267] The automated calibration method for whole-body posture prediction of the sparse IMU described in the present invention, wherein the specific process of calibrating the finger skeleton and the error matrix of the measurement unit deployed on the finger skeleton using the palm-joining action in S8) is as follows:

[0268] The palm-jointing calibration is similar to the Tpose action calibration method. During calibration, the attitude quaternion of each measurement unit in the measuring glove is stored in the cache array, and the Euler angle is used to detect whether there is a large jitter in the calibration action. After the calibration action is completed, the attitude quaternion is taken from the cache array and the effective rotation matrix of the IMU body coordinate system in the geographic coordinate system during the palm-jointing calibration is calculated.

[0269] Among them, g0 is the start time of the palm-joining action, and g1 is the end time of the palm-joining action. The solution process is the same as that of the Tpose action calibration;

[0270] It is known that the rotation matrix of the finger bones relative to the palm back bones in the SMPLH model during the palm-to-palm action is

[0271] A 3×3 ;

[0272] During the Tpose action, the spatial position of the middle finger proximal phalanx joint node in the SMPL coordinate system is obtained according to the SMPLH model: Get the spatial position of the distal phalanx joint node of the middle finger

[0273] So we can find the vector of the middle finger:

[0274]

[0275] Similarly, the vectors V of the other four fingers can be obtained thumb , V index , V ring , V pinky ;

[0276] The thumb vector and index finger vector of the left hand rotate a specific angle θ around the positive direction of the y-axis of the SMPL coordinate system t ,0°<θ t <90°、θ i , 0°<θ i <90°, the left hand's ring finger and little finger are rotated around the negative direction of the y-axis of the SMPL coordinate system by a specific angle θ r , 0°<θ r <90°、θ p , 0°<θ p The included angle between the projection of the index finger vector of the left hand and the projection of the middle finger vector on the xoz plane of the SMPL coordinate system is 45°. The included angle between the projection of the index finger vector, the ring finger vector and the little finger vector on the xoz plane of the SMPL coordinate system and the projection of the middle finger vector on the xoz plane of the SMPL coordinate system is 0°.

[0277] When the palms are together, the rotation matrix of the left thumb phalanx relative to the left palm back bone is:

[0278]

[0279] Similarly, the rotation matrix A of the left index finger phalanges, ring finger phalanges, and little finger phalanges relative to the left palm back bones can be obtained: li 、A lr 、A lp , then the rotation matrix of the left middle finger phalanx relative to the left palm back bone is the identity matrix A lm =I 3×3 ;

[0280] Then, the relative rotation matrix of the phalanges of the right hand relative to the right palm back bones is obtained through mirroring;

[0281] Therefore, the palm-joining action is used to more accurately obtain the error matrix from the bones of the ten fingers to the IMU:

[0282]

[0283] in, The effective rotation matrix of the IMU deployed on the dorsal palm bone in the geographic coordinate system during the palm-closing action;

[0284] in, The effective rotation matrix of the IMU deployed on the finger phalanges in the geographic coordinate system during the palm-joining action;

[0285] in, This is the error matrix from the finger bones to the IMU to be further optimized;

[0286] then This completes the palm-joint calibration.

[0287] Example 2

[0288] The automatic calibration method for whole body posture prediction of a sparse IMU described in this embodiment is the same as that in embodiment 1, except that Figure 4 、 Figure 5 The measurement module and the measurement glove shown include a measurement unit, a WiFi module, a microcontroller unit, and a battery module. The measurement unit and the WiFi module are serially connected to the microcontroller unit. The measurement unit, WiFi module, and microcontroller unit are all connected to the battery module. Each measurement module includes one measurement unit. The measurement glove includes six measurement units, and the measurement units can be distributed as needed. The measurement units include at least one IMU and at least one magnetometer.

[0289] The system also includes a router and a PC host computer. The router is connected to the measurement modules / gloves and the PC host computer. The router is used to network all the measurement modules / gloves and the PC host computer. The router records the physical addresses of all the measurement modules / glove Wi-Fi, assigns a designated IP address, and then sends the data to the PC host computer via a network communication protocol. By improving the structure of the motion capture system, it can use a small number of IMUs to reconstruct the motion posture of fingers and major joints of the human body in real time, improving the accuracy of human motion capture and expanding the application scenarios of IMU-based motion capture. At the same time, the hardware cost of IMUs is much lower than that of motion capture cameras, solving many problems existing in the existing technology.

[0290] The human motion signal captured by the measurement unit is sent to the corresponding microcontroller unit. The inertial signal is preprocessed by the filter in the microcontroller unit, and the inertial signal is fused through the attitude reference system (AHRS) to obtain the rotation attitude (such as Euler angles, attitude quaternions or rotation matrices). The IMU preprocessed data and rotation attitude are then sent to the WiFi module. Finally, these data are sent to the router via a network transmission protocol (such as TCP protocol), and then the data is sent to the PC host computer via a network communication protocol. It is conceivable that in the actual working process, the deployment position of the measurement unit can be adjusted according to actual needs.

[0291] In addition, if Figure 6 As shown, in the data acquisition process in S1) of the automatic calibration method for whole-body posture prediction of a sparse IMU, a data synchronization algorithm is used to ensure that the data of the measurement module processed by the LSTM neural network and the measurement unit in the measurement glove are collected at the same time, thereby improving the accuracy of motion restoration. The data synchronization algorithm is specifically as follows:

[0292] Configure the PC with a specific IP and DNS address to avoid delays caused by dynamic IP allocation;

[0293] Then configure the quality of service, disable bandwidth control, and close unnecessary background applications to minimize network latency between the PC and the router.

[0294] A thread is allocated locally on the PC to run the clock script. The clock script increments the synchronization sequence number sync_id by one every 5 milliseconds. The configured clock script has the characteristics of high precision, real-time, and stability.

[0295] The clock script runs in a dedicated thread locally on the PC, independent of other tasks, thus reducing the possibility of resource contention or delays, and ensuring that the synchronization sequence number sync_id is incremented every 5 milliseconds;

[0296] The PC will wait for the measurement module / glove to connect to the router. When the PC detects that a measurement module / glove is connected to the network, it will send the sync_id to the measurement module / glove. The measurement module / glove will record this sync_id and localize it. The localized sync_id is recorded as sync_id1. The measurement module / glove will increment the localized sync_id1 by one every 5 milliseconds.

[0297] After the PC verifies that all measurement modules / gloves are connected to the network, it stores their communication data in a buffer. Data with consistent sync_ids 1 through 8 are then sent to the LSTM neural network for subsequent algorithm processing. It should be noted that this is the case where the corrected acceleration, angular velocity, magnetometer, and attitude data are sent to the neural network for inference.

[0298] The PC host computer is configured with specific IP and DNS addresses, where specific means that the PC host computer is configured with a static predefined IP address and DNS server address, and Dynamic Host Configuration Protocol (DHCP) is disabled to eliminate IP address negotiation delays; wherein, the IP address is configured as a local area network private address range (such as 192.168.0.0 / 16), and the DNS server address is set to the intranet address of the local router or 127.0.0.1.

[0299] The use of the data synchronization algorithm in this embodiment and the configuration of specific IP and DNS addresses on the PC can effectively avoid the delay caused by dynamic IP allocation, configure quality of service (QoS), disable bandwidth control, and close unnecessary background applications to minimize the network delay between the PC and the router, effectively ensuring that the data of the six measurement modules processed by the neural network are collected at the same time, which helps to improve the accuracy of motion restoration.

[0300] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principles of the present invention. These improvements should also be regarded as the scope of protection of the present invention.

Claims

1. An automated calibration method for whole-body pose prediction using a sparse IMU, characterized by: include: The specific calibration method is as follows: S1): After the user puts on the measurement module / measuring gloves, they face due north and perform a Tpose action; data is collected and preprocessed; S2): AHRS solves and obtains the posture of each measurement unit in the measurement module / measuring glove in the NED coordinate system during the Tpose action, and transmits the posture obtained in the NED coordinate system to the PC host computer through the router. The posture of the measurement unit solved in the i frame in the NED coordinate system is recorded as S3): Automatically obtain the rotation matrix of the SMPL coordinate system and the NED geographic coordinate system through the PC host computer S4): The PC host computer measures the attitude quaternion of each IMU in the module and the inertial acceleration measured by the IMU Measured angular velocity All are stored in the cache array; S5): User Tpose calibration action judgment: If the change amplitude of the Euler angle detected by the IMU exceeds the preset value during the user Tpose calibration action, the automatic calibration is considered to have failed; S6): When the user performs the Tpose calibration action, if the change amplitude of the Euler angle detected in the IMU is less than the preset value and the set calibration time has not been reached, then return to S4); if the change amplitude of the Euler angle detected in the IMU is less than the preset value and the set calibration time has been reached, then calculate the average attitude quaternion of the IMU from the cache array, and the average attitude quaternion of the IMU is converted into a rotation matrix after normalization; S7): Obtain calibration parameters of the accelerometer and gyroscope; S8): Use the palm-joining motion to calibrate the error matrix of the finger skeleton and the measurement unit deployed on the finger skeleton.

2. The automated calibration method for whole-body posture prediction of a sparse IMU according to claim 1, characterized in that: The AHRS solution in S2) obtains the posture of each measurement unit in the measurement module / measuring glove in the NED coordinate system during the Tpose action. A distributed MCU is used to calculate the posture of each measurement unit in the measurement module / measuring glove in the NED coordinate system and calibrate the error of the IMU relative to the human skeleton. The specific calibration process is as follows: The IMU's deployment position is not aligned with the corresponding bones due to muscles, skin or wearing method. Use Tpose / Apose / Stand Attention to calibrate the error of the IMU relative to the human skeleton. The details are as follows: in, is the representation of the i-th frame pose of the skeleton of the deployed measurement unit in the SMPL coordinate system, is the representation of the geographic coordinate system in the SMPL coordinate system, as described in S3), has the properties of an orthogonal matrix, so is the error matrix from the human skeleton to the corresponding IMU.

3. The automated calibration method for whole-body posture prediction of a sparse IMU according to claim 2, characterized in that: During the calibration of the IMU error relative to the human skeleton, the human skeleton posture of all SMPL coordinate systems when posing the Tpose action Both are identity matrices I 3×3 ,Right now in, It is the effective rotation matrix of the IMU body coordinate system in the geographic coordinate system during the Tpose action, t0 is the start time of the Tpose action, and t1 is the end time of the Tpose action; It is the error matrix from human skeleton to IMU, which is a unit orthogonal matrix. The specific solution process is as follows:

4. The automated calibration method for whole-body posture prediction of a sparse IMU according to claim 1, characterized in that: In step S3), the rotation matrix of the SMPL coordinate system and the NED geographic coordinate system is obtained. After the calibration, the IMU attitude in the NED geographic coordinate system needs to be converted to the SMPL coordinate system. The specific calibration process is as follows: The xyz axes of the SMPL coordinate system correspond to the sagittal normal vector in the left direction, the transverse normal vector in the upward direction, and the coronal normal vector in the forward direction, which has a rotation deviation from the northeast geographic coordinate system; The specific rotation deviations are as follows: Among them, v e is the representation of any vector in the geographic coordinate system, v s is the representation of any vector in the SMPL coordinate system.

5. The automated calibration method for whole-body posture prediction of a sparse IMU according to claim 1, characterized in that: In the step S7), the calibration parameters of the accelerometer and gyroscope are obtained, that is, the errors of acceleration and angular velocity are calculated from the cache data set in S4). The error model is as follows: in, is the true inertial acceleration, is the inertial acceleration measured by IMU, T a is the small rotation matrix for accelerometer correction, K a is the accelerometer scale factor, b′ a is the accelerometer bias, ε a is the accelerometer white noise, assuming ε a is a zero mean value with variance σ a Gaussian process; is the true angular velocity, is the angular velocity measured by IMU, T ω is the small rotation matrix of the gyroscope correction, K ω is the gyroscope scale factor, b′ ω is the gyroscope bias, ε ω is the gyroscope white noise.

6. The automated calibration method for whole-body posture prediction of a sparse IMU according to claim 5, characterized in that: During the user's Tpose calibration action in S6), it is assumed that all IMUs are relatively stationary during the Tpose action. Therefore, the least squares method is used to fit the IMU's zero bias, scale factor, and small rotation matrix. Then, the corrected IMU readings are obtained through correction, and the corrected acceleration and angular velocity are obtained. The error parameters of the IMU acceleration and IMU angular velocity are calculated: The objective function of IMU acceleration is: in, is the value of the real inertial acceleration of the i-th frame, and N is the length of the cache array; g is the local acceleration, which is obtained by minimizing J a Fit T a , K a , b′ a , the T a , K a , b′ a They are the small rotation matrix, accelerometer scale factor, and accelerometer bias for accelerometer correction respectively; The objective function of the IMU angular velocity is in, is the value of the true angular velocity of the i-th frame, By minimizing J ω Fit T ω , K ω , b′ ω To obtain more accurate motion acceleration, these calibration parameters are then sent to each MCU via the host PC through the router to obtain more accurate inertial information; Among them, T ω , K ω , b′ ω They are the tiny rotation matrix of gyroscope correction, gyroscope scale factor, and gyroscope zero bias respectively.

7. The automated calibration method for whole-body posture prediction of a sparse IMU according to claim 1, characterized in that: In the above S6), the average attitude quaternion of the IMU is calculated in the cache array, and the specific process of converting the average attitude quaternion into a rotation matrix after normalization is as follows: in, The i-th frame pose quaternion of the IMU deployed at the specified joint position in the cache array. The pose quaternion is the unit pose quaternion rotated from the geographic coordinate system to the IMU body coordinate system; The average attitude quaternion of the IMU deployed at the specified joint position in the cache array; is the normalized average attitude quaternion, its real part is q0, and its three imaginary parts are q1, q2, and q3 respectively; is the effective rotation matrix of the IMU body coordinate system in the geographic coordinate system during the T action, that is, 8. The automated calibration method for whole-body posture prediction of a sparse IMU according to claim 1, characterized in that: In S1), UKF is used to correct the magnetometer error. The magnetometer correction is performed in real time by the UKF algorithm in the MCU. The measurement model of the magnetometer is: B k =(I+D) -1 (A k H k +b+ε k ),k=1,...,N Among them, B k is the measurement value of the magnetic field by the magnetometer at time k, H k is the corresponding value of the geomagnetic field related to the Earth's fixed coordinate system, A k is the unknown attitude matrix of the magnetometer relative to the geographic coordinate system, I is the 3×3 identity matrix, D is the matrix of the magnetometer's scale factor error and non-orthogonal row error, b is the magnetometer's zero bias vector, ε k is the measurement noise vector, assuming that the measurement noise vector is a vector with zero mean and covariance ∑ k Gaussian process; z k The measurement equation is: z k =||B k || 2 -||H k || 2 Expand and organize k Measurement equation: In which, define v k is the error term of the noise part: v k =2((I+D)B k -b) T e k -||e k || 2 v k Approximately Gaussian noise, so its mean μ k and variance for: μ k =E{v k }=E{2((I+D)B k -b) T ε k -||ε k || 2 }=-Tr(∑ k ) Among them, E{} is the expectation of the variable, Tr() is the trace of the variable, σ k It is a function of the matrix D of the magnetometer's scale factor error and non-orthogonal row error and the magnetometer bias vector b. The variables are then iteratively optimized using the UKF. The prediction and correction steps follow the standard unscented Kalman filter method to obtain more accurate magnetometer readings.

9. The automated calibration method for whole-body posture prediction of a sparse IMU according to claim 1, characterized in that: During the calibration process, if the predicted whole-body posture includes fingers, the palm-joining calibration in S8) is performed to optimize the error matrix from the finger bones to the corresponding IMU. Since the five fingers of the palm are straightened in the T action, the angles between adjacent fingers vary from 3 to 45 degrees, making it difficult for the user to perform a standard finger movement. When the five fingers are straightened, the other four fingers except the thumb are parallel, and the thumb and index finger form a 45-degree angle, the palm-joining action is relatively easy for the user to perform, so a more accurate calibration is performed using the palm-joining action. Since the Tpose motion calibration was performed before, the posture of the palm-dorsal bones in the glove measured in the i-th frame in the SMPL coordinate system is in, The posture of the IMU deployed on the back of the palm in the geographic coordinate system at the i-th frame, The error matrix from the palm-dorsal skeleton to the IMU deployed on the palm-dorsal; is calculated as follows: in, The effective rotation matrix of the IMU deployed on the back of the palm in the geographic coordinate system during the T maneuver.

10. The automated calibration method for whole-body posture prediction of a sparse IMU according to claim 1, characterized in that: The specific process of using the palm-joining action to calibrate the finger bones and the error matrix of the measurement unit deployed on the finger bones in S8) is as follows: The palm-jointing calibration is similar to the Tpose action calibration method. During calibration, the attitude quaternion of each measurement unit in the measuring glove is stored in the cache array, and the Euler angle is used to detect whether there is a large jitter in the calibration action. After the calibration action is completed, the attitude quaternion is taken from the cache array and the effective rotation matrix of the IMU body coordinate system in the geographic coordinate system during the palm-jointing calibration is calculated. Among them, g0 is the start time of the palm-joining action, and g1 is the end time of the palm-joining action. The solution process is the same as that of the Tpose action calibration; It is known that when the SMPLH model is in the palm-to-palm action, the rotation matrix A of the finger bones relative to the palm back bones is 3×3 ; During the Tpose action, the spatial position of the middle finger proximal phalanx joint node in the SMPL coordinate system is obtained according to the SMPLH model: Get the spatial position of the distal phalanx joint node of the middle finger So we can find the vector of the middle finger: Similarly, the vectors V of the other four fingers can be obtained thumb , V index , V ring , V pinky ; The thumb vector and index finger vector of the left hand rotate a specific angle θ around the positive direction of the y-axis of the SMPL coordinate system t ,0°<θ t <90°、θ i , 0°<θ i <90°, the left hand's ring finger and little finger are rotated around the negative direction of the y-axis of the SMPL coordinate system by a specific angle θ r , 0°<θ r <90°、θ p , 0°<θ p The included angle between the projection of the index finger vector of the left hand and the projection of the middle finger vector on the xoz plane of the SMPL coordinate system is 45°. The included angle between the projection of the index finger vector, the ring finger vector and the little finger vector on the xoz plane of the SMPL coordinate system and the projection of the middle finger vector on the xoz plane of the SMPL coordinate system is 0°. When the palms are together, the rotation matrix of the left thumb phalanx relative to the left palm back bone is: Similarly, the rotation matrix A of the left index finger phalanges, ring finger phalanges, and little finger phalanges relative to the left palm back bones can be obtained: li 、A lr 、A lp , then the rotation matrix of the left middle finger phalanx relative to the left palm back bone is the identity matrix A lm =I 3×3 ; Then, the relative rotation matrix of the phalanges of the right hand relative to the right dorsal metacarpal bones is obtained through mirroring; Therefore, the palm-joining action is used to more accurately obtain the error matrix from the bones of the ten fingers to the IMU: in, The effective rotation matrix of the IMU deployed on the dorsal palm bone in the geographic coordinate system during the palm-closing action; in, The effective rotation matrix of the IMU deployed on the finger phalanges in the geographic coordinate system during the palm-joining action; in, This is the error matrix from the finger bones to the IMU to be further optimized; then This completes the palm-joint calibration.

Images