IMU calibration method and device for inertial motion capture system, medium and product

Abstract

The invention provides an IMU calibration method and device for an inertial motion capture system, a medium and a product, the inertial motion capture system comprises a plurality of IMUs, a calibration parameter resolving network and an attitude estimation model, the method comprises the following steps: S1, acquiring IMU rotation reading sequences of the plurality of IMUs at regular time; s2, calculating the rotation richness of each IMU rotation reading sequence; s3, inputting the rotation reading sequence of each IMU into a calibration parameter resolving network, estimating the calibration parameter of each IMU through the calibration parameter resolving network, and updating the calibration parameter with the rotation richness higher than a preset threshold value; and S4, calculating the updated calibration parameters and the IMU rotation reading sequences corresponding to the updated calibration parameters to obtain a global joint attitude, and inputting the global joint attitude into the attitude estimation model. By means of the technical scheme, the accuracy of inertial motion capture is improved, meanwhile, the user experience is optimized, and the inertial motion capture system keeps efficient and stable in long-time continuous use.

Description

Technical Field

[0001] The present invention relates to the field of motion capture, and in particular to an IMU calibration method, device, medium and product for an inertial motion capture system. Background Art

[0002] With the widespread application of motion capture technology in virtual reality, game development, sports analysis and other fields, as well as the continuous advancement and popularization of IMU (inertial measurement unit) technology, IMU-based motion capture technology has attracted more and more attention from academia and industry due to its superior ease of use and data privacy. Take Sony Mocopi products as an example. The product contains 6 IMUs and tight straps. Users only need to wear 6 IMUs on designated joints to achieve full-body motion capture. When using such inertial motion capture systems, IMU calibration is an indispensable link, which includes the estimation of the coordinate system drift matrix and the wearing position offset matrix.

[0003] The commonly used IMU calibration currently requires the user to calibrate the IMU coordinate system and perform specific calibration actions (such as T-Pose or N-Pose) according to the prompts when the system starts. The defect of this calibration is that its estimated calibration parameters (coordinate system drift matrix and wearing position offset matrix) cannot be dynamically updated during the use of the system, and the calibration parameters will continue to change during the use of the device due to IMU drift and changes in wearing position, thereby affecting the accuracy of motion capture. In addition, users often need to redo the entire calibration process, which is not only cumbersome but also interrupts the continuous motion capture experience. Therefore, the existing calibration method can only maintain accuracy for a limited time, which limits the application of inertial motion capture systems in long-term use scenarios. Summary of the invention

[0004] The embodiments of the present invention provide an IMU calibration method, device, medium and product for an inertial motion capture system, aiming to improve the accuracy of inertial motion capture while optimizing the user experience so that the inertial motion capture system remains efficient and stable during long-term continuous use.

[0005] In order to achieve the above object, on the one hand, an IMU calibration method for an inertial motion capture system is provided, wherein the inertial motion capture system includes a plurality of IMUs, a calibration parameter solution network, and a posture estimation model, and the method includes:

[0006] Step S1, regularly acquiring measurement data of a plurality of IMUs at a preset time interval, wherein the measurement data comprises a sequence of IMU rotation readings of a predetermined length;

[0007] Step S2, calculating the rotation richness of the IMU rotation reading sequence of each IMU, wherein the rotation richness of the IMU rotation reading sequence is: dividing the continuous Euler angle space according to a predetermined angle, and then converting it into a discrete space of a predetermined size, and counting the number of rotation readings in the IMU rotation reading sequence greater than 0 in the discrete space;

[0008] Step S3, inputting the IMU rotation reading sequence of each IMU into the calibration parameter solution network, obtaining an estimated value of the calibration parameter of each IMU through the calibration parameter solution network, and using the estimated value of the calibration parameter to update the calibration parameters of the IMU whose rotation richness is higher than a preset threshold, wherein the correction parameters include: a global coordinate system drift matrix and a wearing position offset matrix, and the calibration parameter solution network includes: an encoder and a Transformer pooling mapping TPM module;

[0009] The calibration parameter solution network is modeled as follows:

[0010] f Glob (·)=TPM Glob (E(·))

[0011] f Loca (·)=TPM Loca (E(·))

[0012] f Glob represents the drift matrix solution function of the global coordinate system, f Loca Represents the wear position offset matrix solution function, TPM Glob represents a TPM module for solving the drift matrix of the global coordinate system, TPM Loca represents a TPM module for solving the wearing position offset matrix, and E represents the encoder;

[0013] Step S4, calculating according to the updated correction parameters and the corresponding IMU rotation reading sequence to obtain the corresponding global joint posture, and inputting the global joint posture into the posture estimation model;

[0014] The global joint posture at the corresponding time t is calculated by the following formula:

[0015]

[0016] R IMU represents the first IMU rotation reading sequence, represents the global coordinate system drift matrix, represents the global joint posture, represents the wearing position offset matrix, G is the predetermined ideal world coordinate system, and G* is the world coordinate system of the real IMU.

[0017] Preferably, in the IMU calibration method for an inertial motion capture system, when the number of IMUs is greater than a preset number, the measurement data further includes: IMU acceleration measurement values;

[0018] The step S4 further comprises: calculating the global acceleration at the corresponding time t using the following formula according to the IMU acceleration measurement value, and inputting the global acceleration into the attitude estimation model, wherein:

[0019]

[0020] a IMU Represents the IMU acceleration measurement value, a G represents the global acceleration, represents the global coordinate system drift matrix.

[0021] Preferably, in the IMU calibration method for an inertial motion capture system, the predetermined number is 6.

[0022] Preferably, the IMU calibration method for an inertial motion capture system, wherein the calibration parameter solution network is implemented based on a Transformer model, wherein the encoder is a network composed of multiple layers of Transformer encoder modules, used to extract the time-space features of the first IMU rotation reading sequence; the TPM module is composed of a layer of Transformer encoder, time average pooling, and a layer of linear mapping.

[0023] Preferably, in the IMU calibration method for an inertial motion capture system, the calibration parameter solution network uses random samples dynamically generated from a predetermined large-scale motion capture data set for model training, wherein:

[0024] Fully calibrated IMU data is obtained through human motion data in the large-scale motion capture data set, and a random global coordinate system drift matrix and a wearing position offset matrix are added to the fully calibrated IMU data to generate training data, wherein the fully calibrated IMU data can simulate accurate skeletal posture, and the global coordinate system drift matrix and the wearing position offset matrix are randomly sampled from a pre-set uniformly distributed Euler angle space.

[0025] Preferably, in the IMU calibration method for an inertial motion capture system, the calibration parameter solution network is trained using a supervised learning method, and the supervised learning method uses a loss function that combines a correction error loss function and a priori loss function during training, wherein:

[0026] The correction error loss function is:

[0027]

[0028] L cali represents the correction error loss function, represents the IMU rotation readings in the generated training data, represents the global coordinate drift matrix corresponding to the generated training data, Represents the wearing position offset matrix corresponding to the generated training data, represents the square of the L2 norm, and n represents the predetermined length of the IMU rotation reading sequence;

[0029] The prior loss function is:

[0030]

[0031] L prior represents the prior loss function, represents a calibrated IMU rotation reading, I represents a unit matrix, and n represents a predetermined length of the IMU rotation reading sequence;

[0032] The loss function of the correction error loss function combined with the prior loss function is:

[0033] L all =L cali +L prior

[0034] L all Represents the loss function used by the calibration parameter solving network.

[0035] In another aspect, an inertial motion capture system is provided, wherein the inertial motion capture system implements any of the calibration methods described above during operation, comprising:

[0036] Multiple IMUs, used to obtain IMU measurement data;

[0037] The calibration parameter solving network is used to solve the IMU rotation reading sequence in the acquired IMU measurement data;

[0038] The posture estimation model is used to simulate the posture based on the acquired global joint posture and global acceleration.

[0039] On the other hand, an IMU calibration device for inertial motion capture includes a memory and a processor, wherein the memory stores at least one program, and the at least one program is executed by the processor to implement any of the calibration methods described above.

[0040] On the other hand, a computer-readable storage medium is provided, wherein at least one program is stored in the storage medium, and the at least one program is executed by a processor to implement any calibration method as described above.

[0041] In yet another aspect, a computer program product comprises a computer program, wherein when the computer program is executed by a processor, the calibration method as described above is implemented.

[0042] The above technical solution has the following technical effects:

[0043] The technical solution of this embodiment obtains the measurement data of multiple IMUs at a preset time interval. When the rotation diversity (RD) of the IMU rotation reading sequence in the measurement data is greater than a preset threshold, the rotation reading sequence in the IMU measurement data is input into a pre-trained calibration parameter solution network (TIC) network, and the global coordinate system drift matrix and the wearing position offset matrix in the IMU rotation reading sequence are solved by the calibration parameter solution network to obtain the estimated value of the calibration parameter, and then the estimated value of the correction parameter is used to calculate with the measurement data to obtain the global joint posture and input it into the posture estimation model. The intelligent dynamic calibration method of the above technical solution not only significantly improves the stability and accuracy of the inertial motion capture system in long-term use, but also reduces unnecessary calibration operations, avoids calibration when the rotation information is insufficient, thereby reducing the error accumulation caused by invalid updates, optimizing the calibration process, and reducing the dependence on user-specific calibration actions, optimizing the user experience.

[0044] In a further embodiment, the calibration parameter solution network is constructed based on the Transformer model, and the model is trained using purely synthetic data from the large-scale motion capture (AMASS) dataset. This allows the technical solution of this embodiment to be easily extended to different inertial motion capture systems, greatly improving the practicality, economy and adaptability of the inertial motion capture system.

[0045] In a further embodiment, the calibration parameter solution network uses a loss function that combines a calibration error loss function with a priori loss function to perform supervised learning, thereby improving the accuracy and robustness of the calibration parameter solution network in estimating calibration parameters.

[0046] In a further embodiment, by dividing the continuous Euler angle space according to a preset angle, and then converting the Euler angle space into a discrete space, and further calculating the rotation richness of the IMU rotation reading sequence, the rotation richness can be quantified more accurately, thereby improving the reliability of the calibration parameter estimation; BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A schematic flow chart of an IMU calibration method for an inertial motion capture system according to an embodiment of the present invention;

[0048] Figure 2 This is a flow chart of an IMU dynamic calibration of an IMU calibration method for an inertial motion capture system according to an embodiment of the present invention;

[0049] Figure 3 FIG. 1 is a diagram of an IMU calibration device for inertial motion capture according to an embodiment of the present invention. DETAILED DESCRIPTION

[0050] To further illustrate the various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, a person of ordinary skill in the art should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0051] The present invention will now be further described with reference to the accompanying drawings and specific implementation methods.

[0052] Embodiment 1:

[0053] In order to improve the accuracy of inertial motion capture and optimize the user experience so that the inertial motion capture system can maintain high efficiency and stability during long-term continuous use, this embodiment proposes an IMU calibration method for an inertial motion capture system. Figure 1 FIG. 1 is a flow chart of an IMU calibration method for an inertial motion capture system according to an embodiment of the present invention. Figure 1 As shown, the inertial motion capture system includes multiple IMUs, a calibration parameter solution network, and a posture estimation model. The method includes:

[0054] Step S1, regularly acquiring measurement data of a plurality of IMUs at a preset time interval, wherein the measurement data includes an IMU rotation reading sequence of a predetermined length;

[0055] Step S2, calculating the rotation richness of the IMU rotation reading sequence of each IMU, wherein the rotation richness of the IMU rotation reading sequence is: dividing the continuous Euler angle space according to a predetermined angle, and then converting it into a discrete space of a predetermined size, and counting the number of rotation readings in the IMU rotation reading sequence greater than 0 in the discrete space;

[0056] Step S3, inputting the IMU rotation reading sequence of each IMU into the calibration parameter solution network, obtaining the estimated value of the calibration parameter of each IMU through the calibration parameter solution network, and using the estimated value of the calibration parameter to update the calibration parameters of the IMU whose rotation richness is higher than a preset threshold, wherein the correction parameters include: a global coordinate system drift matrix and a wearing position offset matrix, and the calibration parameter solution network includes: an encoder and a Transformer pooling mapping TPM module;

[0057] Among them, the calibration parameter solution network is modeled as follows:

[0058] f Glob (·)=TPM Glob (E(·))

[0059] f Loca (·)=TPM Loca (E(·))

[0060] f Glob represents the drift matrix solution function of the global coordinate system, f Loca Represents the wear position offset matrix solution function, TPM Glob Represents the TPM module used to solve the drift matrix of the global coordinate system. Loca represents the TPM module used to solve the wear position offset matrix, and E represents the encoder;

[0061] Step S4, calculating according to the updated correction parameters and the corresponding IMU rotation reading sequence to obtain the corresponding global joint posture, and inputting the global joint posture into the posture estimation model;

[0062] Among them, the global joint posture corresponding to time t is calculated by the following formula:

[0063]

[0064] R IMU represents the first IMU rotation reading sequence, represents the global coordinate system drift matrix, represents the global joint pose, Represents the wear position offset matrix, G is the predetermined ideal world coordinate system, and G* is the world coordinate system of the real IMU.

[0065] Embodiment 2:

[0066] During the use of IMU, coordinate system drift and wearing position offset may occur, resulting in dynamic changes in calibration parameters. However, existing methods cannot update calibration parameters during motion capture. Once the IMU drifts or changes position, the user must put on the device again and calibrate again, resulting in interruption of the motion capture process. Therefore, the technical solution of this embodiment can update the calibration parameters in real time during the motion capture process, effectively solving the problem of accumulated posture estimation errors caused by IMU drift and wearing position offset during long-term use, and there is no need for the user to put on or take off the device again or maintain specific calibration actions, thus ensuring a continuous user experience.

[0067] Figure 2 The IMU dynamic calibration flow chart of the IMU calibration method for an inertial motion capture system according to an embodiment of the present invention. The technical solution of this embodiment uses the Transformer model to realize the calibration parameter estimation. The model takes the real-time IMU rotation reading sequence collected by the inertial motion capture system as input and outputs the estimated value of the calibration parameter. At the same time, the technical solution uses a calibration parameter update strategy based on rotation diversity (RD), and calibration is performed only when the rotation diversity is higher than a threshold, such as Figure 2 As shown, the method includes:

[0068] Step 1, regularly obtaining the latest rotation reading sequence of each IMU of a predetermined length according to a predetermined time interval; preferably, the time interval is 1 second; preferably, the predetermined length is 128 frames; preferably, for a sparsely arranged inertial motion capture system, for example, only using 6 IMUs, the global acceleration measurement value measured by the IMU will be used as input;

[0069] The global joint pose required by the inertial motion capture system is The global acceleration is a G , then the original IMU attitude and acceleration measurement values ​​can be expressed as:

[0070]

[0071]

[0072] Among them, R IMU represents the IMU rotation reading sequence, a IMU Represents the IMU acceleration measurement value, represents the global joint posture, a G represents the global acceleration, represents the global coordinate system drift matrix, represents the wear position offset matrix, t represents the corresponding time such as moment, G is the predetermined ideal world coordinate system, and G* is the world coordinate system of the real IMU.

[0073] The goal of IMU calibration is to accurately solve the above two matrices, so as to convert the original IMU rotation reading sequence into the input required by the inertial motion capture system. From the above formula, it can be seen that the IMU rotation reading sequence is the calibration parameter such as and and skeletal movement such as The core goal of the dynamic calibration in the technical solution of this embodiment is to construct two functions f Glob With f Loca , so that the two can respectively and Decoupled from the raw IMU readings.

[0074] Step 2, calculate the rotation richness of the IMU rotation reading sequence of each IMU;

[0075] Calibration Parameter Solution The accurate solution of the calibration parameters by the TIC network depends on the rotation richness of the input IMU rotation reading sequence, because the rotation readings of the IMU under different postures can provide richer solution constraints and narrow the solution space of the calibration parameters. For this reason, it is hoped that calibration will be triggered only when the rotation richness is sufficient.

[0076] In a specific implementation, in order to achieve the quantification of rotation diversity, the rotation diversity (RD) of the IMU rotation reading sequence is calculated as shown in Table 1 below. The IMU rotation reading sequence R with a predetermined length n is input. IMU (1)…R IMU (n), the continuous Euler angle space (θ x ∈[-180, 180],θ y ∈[-90, 90],θ z ∈[-180, 180]) is divided into every predetermined angle such as 15 degrees and converted into a 24×12×24 discrete space S. Each element in the discrete space is initialized to 0. Then, the input IMU rotation reading sequence is traversed and each reading R is converted into IMU (t) is mapped to the corresponding position i, j, k in S, and the corresponding positions are counted cumulatively, where t ranges from 1 to n. Finally, the rotation richness RD is obtained by calculating the number of elements greater than zero in S, where Represents the universal quantifier in the logical operator, Represents three-dimensional real number space.

[0077]

[0078] Table 1

[0079] Step 3, inputting the IMU rotation reading sequence of each IMU into the calibration parameter solution network, obtaining the estimated value of the calibration parameter of each IMU through the calibration parameter solution network, and using the estimated value of the calibration parameter to update the calibration parameter of the IMU whose rotation richness is higher than a preset threshold;

[0080] In one specific implementation, the input data of the TIC network can be replaced by a combination of an IMU rotation reading sequence and other measurable data, such as acceleration, magnetometer readings, etc.

[0081] Preferably, to construct f Glob With f Loca The technical solution of this embodiment proposes a calibration parameter solution network TIC, which is implemented based on the Transformer neural network model and consists of two components:

[0082] 1) Encoder (E): A network consisting of 3 layers of Transformer encoder modules for temporal-spatial feature extraction;

[0083] 2) Transformer-Pooling-Mapping (TPM) module: A module consisting of a layer of Transformer encoder, temporal average pooling, and a layer of linear mapping.

[0084] In a specific implementation, the TIC network can be replaced by other types of neural networks, such as convolutional neural networks, recurrent neural networks, etc.

[0085] In a specific implementation, the TIC network contains two TPM modules, namely, TPM for solving the drift matrix of the global coordinate system Glob TPM for wear position offset matrix solution Loca ;

[0086] Therefore, the TIC network will Glob With f Loca The modeling is as follows:

[0087] f Glob (·)=TPM Glob (E(·))

[0088] f Loca (·)=TPM Loca (E(·))

[0089] (·) indicates input parameters of the same dimension;

[0090] In a specific implementation, for a TIC network suitable for a 6-IMU inertial motion capture system, the structural details are shown in Table 2 below:

[0091] Parameter name Numeric Input Dimensions 54 Number of Encoder Layers 3 Token Dimension 256 Multi-head attention 8 Feedforward network hidden layer dimensions 512 Output Dimensions 36+36

[0092] Table 2

[0093] The TIC network uses the attitude measurement value sequence of 6 IMU rotation matrix format as input and outputs the calibration parameters of each of the 6 IMUs. and in, and All are in R6D format.

[0094] Preferably, the technical solution of this embodiment uses random samples dynamically generated based on the large-scale motion capture (AMASS) dataset for model training; in a specific implementation, firstly, fully calibrated IMU data is obtained through human motion data in the AMASS dataset. Fully calibrated IMU data It can simulate accurate skeleton posture; the length of each training sample is 128 frames of data at 30Hz, corresponding to a time length of 4.26 seconds; further, Add a random global coordinate system drift matrix Offset matrix from wearing position To generate training data Preferably, and It is obtained by random sampling from the uniformly distributed Euler angle space. The random sampling space of the Euler angle space is shown in Table 3 below:

[0095]

[0096] Table 3

[0097] Preferably, since the synthetic training data contains and However, the sample does not include all possible situations, so in actual use, the update of the calibration parameters will be carried out in the form of variation estimation to ensure that the predicted and It is always within the distribution range of the training data, so that the technical solution of this embodiment can adapt to all possible sensor drifts and wearing position offsets.

[0098] Preferably, the TIC network is trained using supervised learning, and the loss function used is a loss function L that combines the calibration error loss function with the prior loss function. all ; Calibration error loss function L cali , defined as:

[0099]

[0100] in, represents the IMU rotation readings in the generated training data, represents the global coordinate drift matrix corresponding to the generated training data, Represents the wearing position offset matrix corresponding to the generated training data, represents the square of the L2 norm, and n represents the predetermined length of the IMU rotation reading sequence; the calibration error loss function will guide the neural network to learn how to separate the calibration parameters that can be regarded as fixed values ​​in a short time from the raw IMU readings;

[0101] In addition to the calibration error loss, the technical solution of this embodiment also adopts an additional priori loss function L prior For TIC network training, where:

[0102]

[0103] is the calibrated IMU rotation reading, and I represents the identity matrix. The prior loss function will guide the neural network to estimate the calibration parameters as the identity matrix when the fully calibrated data is input, providing further constraints for the training of the neural network.

[0104] The final loss function used is L cali With L prior Combination of:

[0105] L all =L cali +L prior

[0106] Preferably, the data batch size during model training is 256, the learning rate is 0.001, the Adam optimizer is used, and the training epoch is 20.

[0107] Step 4, calculate according to the updated correction parameters and the corresponding IMU rotation reading sequence to obtain the global joint posture, and input the global joint posture into the posture estimation model; in a specific implementation, it also includes: calculate according to the estimated value of the correction parameter and the acceleration measurement value to obtain the global acceleration, and input the global acceleration into the posture estimation model.

[0108] In a specific implementation, based on the rotation richness calculation of the trained TIC network and the IMU rotation reading sequence, the dynamic IMU calibration of the technical solution of this embodiment can be synchronized with the inertial motion capture system. Table 4 below shows the specific algorithm of the dynamic IMU calibration of inertial motion capture. Specifically, the input data of the algorithm is the uncalibrated IMU rotation reading R IMU (1), R IMU (2), ... and the acceleration reading a of the IMU IMU(1) , a IMU(2) , ..., the return value is the calibrated IMU data Initialize the data index i to 1 and create a data buffer B with a maximum length of n n , and initialized with the identity matrix and Preferably, the maximum length n is a predetermined length of the IMU rotation reading sequence;

[0109] Each loop will take a new IMU rotation reading from the uncalibrated IMU rotation reading and pass it through and Calibrate and add to B n middle;

[0110] When B n When the number of rotation readings added reaches the maximum length, calculate B n The rotational richness RD of the mid-rotated read sequence;

[0111] If RD is greater than the preset threshold T R , it is considered that there is enough rotation change to perform a calibration. At this time, the TIC network is called to obtain the correction value and and update and

[0112] Then, clear B n , the data index is incremented and prepares for the next round of data collection.

[0113]

[0114] Table 4

[0115] In summary, the IMU calibration currently used in inertial motion capture systems relies on fixed calibration parameters calculated through specific calibration actions when the system starts, and cannot adapt to the dynamic changes of calibration parameters during subsequent use. The technical solution of this embodiment uses the IMU readings during the inertial motion capture process to achieve real-time estimation and update of calibration parameters, effectively ensuring the accuracy of posture estimation of the inertial motion capture system during long-term use. At the same time, the calibration parameter estimation model used in the technical solution of this embodiment can be trained using purely synthetic data, which allows this solution to be easily extended to inertial motion capture systems with different IMU settings.

[0116] Embodiment three:

[0117] The present invention also provides an IMU calibration device for inertial motion capture, such as Figure 3 As shown, the device includes a processor 301, a memory 302, a bus 303, and a computer program stored in the memory 302 and executable on the processor 301. The processor 301 includes one or more processing cores. The memory 302 is connected to the processor 301 via the bus 303. The memory 302 is used to store program instructions. When the processor executes the computer program, the steps in the above method embodiment of the first embodiment of the present invention are implemented.

[0118] Further, as an executable solution, the IMU calibration device for inertial motion capture can be a computer unit, which can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The computer unit may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that the composition structure of the above-mentioned computer unit is only an example of a computer unit and does not constitute a limitation on the computer unit. It may include more or less components than the above, or a combination of certain components, or different components. For example, the computer unit may also include input and output devices, network access devices, buses, etc., which are not limited in the embodiments of the present invention.

[0119] Further, as an executable solution, the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer unit, and various interfaces and lines are used to connect various parts of the entire computer unit.

[0120] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the computer unit by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required for a function; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0121] Embodiment 4:

[0122] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method in the embodiment of the present invention are implemented.

[0123] If the module / unit integrated in the computer unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory) and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0124] Embodiment five:

[0125] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method described above are implemented.

[0126] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.

Claims

1. An IMU calibration method for an inertial motion capture system, characterized in that: The inertial motion capture system includes multiple IMUs, a calibration parameter solution network, and a pre-built attitude estimation model, including: Step S1, regularly acquiring measurement data of a plurality of IMUs at a preset time interval, wherein the measurement data comprises a sequence of IMU rotation readings of a predetermined length; Step S2, calculating the rotation richness of the IMU rotation reading sequence of each IMU, wherein the rotation richness of the IMU rotation reading sequence is: dividing the continuous Euler angle space according to a predetermined angle, and then converting it into a discrete space of a predetermined size, and counting the number of rotation readings in the IMU rotation reading sequence greater than 0 in the discrete space; Step S3, inputting the IMU rotation reading sequence of each IMU into the calibration parameter solution network, obtaining an estimated value of the calibration parameter of each IMU through the calibration parameter solution network, and using the estimated value of the calibration parameter to update the calibration parameters of the IMU whose rotation richness is higher than a preset threshold, wherein the correction parameters include: a global coordinate system drift matrix and a wearing position offset matrix, and the calibration parameter solution network includes: an encoder and a Transformer pooling mapping TPM module; The calibration parameter solution network is modeled as follows: f Glob (·)=TPM Glob (E(·)) f Loca (·)=TPM Loca (E(·)) f Glob represents the drift matrix solution function of the global coordinate system, f Loca Represents the wear position offset matrix solution function, TPM Glob represents a TPM module for solving the drift matrix of the global coordinate system, TPM Loca represents a TPM module for solving the wearing position offset matrix, and E represents the encoder; Step S4, calculating according to the updated correction parameters and the corresponding IMU rotation reading sequence to obtain the corresponding global joint posture, and inputting the global joint posture into the posture estimation model; The global joint posture at the corresponding time t is calculated by the following formula: R IMU represents the first IMU rotation reading sequence, represents the global coordinate system drift matrix, represents the global joint posture, represents the wearing position offset matrix, G is the predetermined ideal world coordinate system, and G* is the world coordinate system of the real IMU.

2. The IMU calibration method according to claim 1, characterized in that: When the number of IMUs is greater than a preset number, the measurement data further includes: IMU acceleration measurements; The step S4 further comprises: calculating the global acceleration at the corresponding time t using the following formula according to the IMU acceleration measurement value, and inputting the global acceleration into the attitude estimation model, wherein: a IMU Represents the IMU acceleration measurement value, a G represents the global acceleration, represents the global coordinate system drift matrix.

3. The IMU calibration method according to claim 2, characterized in that: The predetermined number is 6.

4. The IMU calibration method according to claim 1, characterized in that: The calibration parameter solution network is implemented based on the Transformer model, wherein the encoder is a network composed of multiple layers of Transformer encoder modules, which is used to extract the time-space features of the first IMU rotation reading sequence; the TPM module is composed of a layer of Transformer encoder, time average pooling and a layer of linear mapping.

5. The IMU calibration method according to claim 1, characterized in that: The calibration parameter solving network uses random samples dynamically generated from a predetermined large-scale motion capture dataset for model training, wherein: Fully calibrated IMU data is obtained through human motion data in the large-scale motion capture data set, and a random global coordinate system drift matrix and a wearing position offset matrix are added to the fully calibrated IMU data to generate training data, wherein the fully calibrated IMU data can simulate accurate skeletal posture, and the global coordinate system drift matrix and the wearing position offset matrix are randomly sampled from a pre-set uniformly distributed Euler angle space.

6. The IMU calibration method according to claim 5, characterized in that: The calibration parameter solution network is trained using a supervised learning method, and the supervised learning method uses a loss function that combines a correction error loss function and a priori loss function during training, wherein: The correction error loss function is: L cali represents the correction error loss function, represents the IMU rotation readings in the generated training data, represents the global coordinate drift matrix corresponding to the generated training data, Represents the wearing position offset matrix corresponding to the generated training data, represents the square of the L2 norm, and n represents the predetermined length of the IMU rotation reading sequence; The prior loss function is: L prior represents the prior loss function, represents a calibrated IMU rotation reading, I represents a unit matrix, and n represents a predetermined length of the IMU rotation reading sequence; The loss function of the correction error loss function combined with the prior loss function is: L all =L cali +L prior L all Represents the loss function used by the calibration parameter solving network.

7. An inertial motion capture system, characterized in that: The inertial motion capture system implements the IMU calibration method according to any one of claims 1 to 6 during operation, including: Multiple IMUs, used to obtain IMU measurement data; The calibration parameter solving network is used to solve the IMU rotation reading sequence in the acquired IMU measurement data; The posture estimation model is used to simulate the posture based on the acquired global joint posture and global acceleration.

8. An IMU calibration device for inertial motion capture, characterized in that: It includes a memory and a processor, the memory stores at least one program, and the at least one program is executed by the processor to implement the IMU calibration method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The storage medium stores at least one program, and the at least one program is executed by a processor to implement the IMU calibration method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the IMU calibration method according to any one of claims 1 to 6 is implemented.

Images