Imu calibration method, apparatus, medium, and product for inertial motion capture system

Abstract

The application provides an IMU calibration method, device, medium and product for an inertial motion capture system, wherein the inertial motion capture system comprises a plurality of IMUs, a calibration parameter solving network and a pose estimation model, the method comprising: S1, obtaining IMU rotation reading sequences of the plurality of IMUs at a fixed time; S2, calculating rotation richness of each IMU rotation reading sequence; S3, inputting each IMU rotation reading sequence into the calibration parameter solving network, estimating calibration parameters of each IMU through the calibration parameter solving network, and updating the calibration parameters whose rotation richness is higher than a predetermined threshold; S4, calculating the updated calibration parameters and the IMU rotation reading sequences corresponding to the updated calibration parameters to obtain global joint poses, and inputting the global joint poses into the pose estimation model. By using the above technical scheme, the accuracy of inertial motion capture is improved, the user experience is optimized, and the inertial motion capture system remains efficient and stable in long-time continuous use.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of motion capture, in particular to an IMU calibration method, device, medium and product for an inertial motion capture system. BACKGROUND

[0002] With the wide application of motion capture technology in virtual reality, game development, sports analysis and other fields, and the continuous progress and popularization of IMU (Inertial Measurement Unit) technology, the motion capture technology based on IMU is increasingly concerned by the academic and industrial circles due to its superior convenience and data privacy. Taking Sony Mocopi product as an example, the product contains 6 IMUs and a tight bandage, and the user only needs to wear the 6 IMUs on the specified joints to realize the motion capture of the whole body. In the use of such inertial motion capture system, IMU calibration is an indispensable link, which includes the estimation of coordinate system drift matrix and wearing position offset matrix.

[0003] The commonly used IMU calibration currently needs the user to perform IMU coordinate system calibration and make specific calibration actions (such as T-Pose or N-Pose) according to the prompts at the start of the system. The defect of this calibration is that the estimated calibration parameters (coordinate system drift matrix and wearing position offset matrix) cannot be dynamically updated during the use of the system, and the calibration parameters will change constantly in the use of the device due to IMU drift and wearing position change, thereby affecting the accuracy of motion capture. In addition, the user often needs to perform the entire calibration process again, which is not only cumbersome but also interrupts the continuous motion capture experience. Therefore, the existing calibration method can only maintain accuracy for a limited time, limiting the application of the inertial motion capture system in long-time use scenarios. SUMMARY

[0004] Embodiments of the present application provide an IMU calibration method, device, medium and product for an inertial motion capture system, aiming to improve the accuracy of inertial motion capture and optimize the user experience, so that the inertial motion capture system remains efficient and stable in long-time continuous use.

[0005] To achieve the above-mentioned purpose, in one aspect, an IMU calibration method for an inertial motion capture system is provided, the inertial motion capture system comprising a plurality of IMUs, a calibration parameter solving network and a pose estimation model, the method comprising:

[0006] Step S1, obtaining measurement data of the plurality of IMUs at a predetermined time interval, wherein the measurement data comprises a sequence of IMU rotation readings of a predetermined length;

[0007] Step S2, calculating the rotation richness of the IMU rotation reading sequence of each IMU, wherein the rotation richness of the IMU rotation reading sequence is: dividing the continuous Euler angle space according to a predetermined angle, and then converting it into a discrete space of a predetermined size, and counting the number of rotation readings greater than 0 in the IMU rotation reading sequence in the discrete space;

[0008] Step S3, inputting the IMU rotation reading sequence of each IMU into the calibration parameter solving network, obtaining the estimated value of the calibration parameter of each IMU through the calibration parameter solving network, and updating the calibration parameter of the IMU whose rotation richness is higher than a pre-set threshold value using the estimated value of the calibration parameter, wherein the calibration parameter includes: a global coordinate system drift matrix and a wearing position offset matrix, and the calibration parameter solving network includes: an encoder and a Transformer pooling mapping (TPM) module;

[0009] Wherein, the calibration parameter solving network is modeled as follows:

[0010] f Glob (·)=TPM Glob (E(·))

[0011] f Loca (·)=TPM Loca (E(·))

[0012] f Glob represents a global coordinate system drift matrix solving function, f Loca represents a wearing position offset matrix solving function, TPM Glob represents a TPM module for solving the global coordinate system drift matrix, TPM Loca represents a TPM module for solving the wearing position offset matrix, E represents the encoder;

[0013] Step S4, calculating the corresponding global joint pose according to the updated correction parameter and the IMU rotation reading sequence corresponding thereto, and inputting the global joint pose into the pose estimation model;

[0014] Wherein, the global joint pose at the corresponding time t is calculated by the following formula:

[0015]

[0016] R IMU represents the first IMU rotation reading sequence, represents the global coordinate system drift matrix, represents the global joint pose, represents the wearing position offset matrix, G is a predetermined ideal world coordinate system, and G* is a real IMU world coordinate system.

[0017] Preferably, the IMU calibration method for an inertial motion capture system, wherein when the number of IMUs is greater than a predetermined number, the measurement data further comprises: IMU acceleration measurement values;

[0018] The step S4 further comprises: calculating a global acceleration at a corresponding time t according to the IMU acceleration measurement values, using the following formula, and inputting the global acceleration into the pose estimation model, wherein:

[0019]

[0020] a IMU represents the IMU acceleration measurement value, a G represents the global acceleration, represents the global coordinate system drift matrix.

[0021] Preferably, the IMU calibration method for an inertial motion capture system, wherein the predetermined number is 6.

[0022] Preferably, the IMU calibration method for an inertial motion capture system, wherein the calibration parameter solving network is implemented based on a Transformer model, wherein the encoder is a network composed of a plurality of Transformer encoder modules, used to extract time-space features of the first IMU rotation reading sequence; the TPM module is composed of a layer of Transformer encoder, time average pooling and a layer of linear mapping.

[0023] Preferably, the IMU calibration method for an inertial motion capture system, wherein the calibration parameter solving network uses random samples dynamically generated from a predetermined large-scale motion capture data set for model training, wherein:

[0024] Fully calibrated IMU data is obtained through human motion data in the large-scale motion capture data set, and random global coordinate system drift matrix and wearing position offset matrix are added to the fully calibrated IMU data to generate training data, wherein the fully calibrated IMU data can simulate accurate bone posture, and the global coordinate system drift matrix and the wearing position offset matrix are randomly sampled from a predetermined uniform distribution of Euler angle space.

[0025] Preferably, the IMU calibration method for inertial motion capture system, wherein the calibration parameter solving network is trained in a supervised learning manner, and a loss function combining a correction error loss function and a prior loss function is used in the training of the supervised learning manner, wherein:

[0026] The correction error loss function is:

[0027]

[0028] L cali denotes the correction error loss function, denotes the IMU rotation readings in the generated training data, denotes the global coordinate drift matrix corresponding to the generated training data, denotes the wearing position offset matrix corresponding to the generated training data, denotes the square of the L2 norm, and n denotes a predetermined length of the IMU rotation reading sequence;

[0029] The prior loss function is:

[0030]

[0031] L prior denotes the prior loss function, denotes the calibrated IMU rotation readings, I denotes a unit matrix, and n denotes a predetermined length of the IMU rotation reading sequence;

[0032] The loss function combining the correction error loss function and the prior loss function is:

[0033] L all = L cali + L prior

[0034] L all denotes the loss function used by the calibration parameter solving network.

[0035] In another aspect, an inertial motion capture system, wherein the inertial motion capture system implements the calibration method according to any one of the above when running, comprising:

[0036] a plurality of IMUs for acquiring IMU measurement data;

[0037] a calibration parameter solving network for solving an IMU rotation reading sequence in the acquired IMU measurement data;

[0038] a pose estimation model for simulating a pose according to the acquired global joint pose and global acceleration.

[0039] In yet another aspect, an IMU calibration device for inertial motion capture, comprising a memory and a processor, wherein the memory stores at least one program, and the at least one program is executed by the processor to implement the calibration method according to any one of the above aspects.

[0040] In yet another aspect, a computer-readable storage medium, wherein the storage medium stores at least one program, and the at least one program is executed by a processor to implement the calibration method according to any one of the above aspects.

[0041] In yet another aspect, a computer program product, comprising a computer program, wherein the computer program is executed by a processor to implement the calibration method according to any one of the above aspects.

[0042] The above technical solutions have the following technical effects:

[0043] The technical solution of the embodiment acquires the measurement data of the plurality of IMUs at a preset time interval, and when the rotation diversity (RD) of the IMU rotation reading sequence in the measurement data is greater than a preset threshold, inputs the rotation reading sequence in the IMU measurement data into a pre-trained calibration parameter solving network (TIC) network, solves the global coordinate system drift matrix and the wearing position offset matrix in the IMU rotation reading sequence through the calibration parameter solving network, obtains the estimated value of the calibration parameter, and then calculates the estimated value of the correction parameter with the measurement data to obtain the global joint posture and input it into the posture estimation model. The intelligent dynamic calibration method of the above technical solution not only significantly improves the stability and accuracy of the inertial motion capture system in long-term use, but also reduces unnecessary calibration operations, avoids calibration in the case of insufficient rotation information, thereby reducing the error accumulation caused by invalid updates, optimizes the calibration process, and reduces the dependence on user-specific calibration actions, thereby optimizing the user experience.

[0044] In further embodiments, the calibration parameter solving network is constructed based on a Transformer model, and the model is trained using pure synthetic data of a large-scale motion capture (AMASS) dataset, which makes the technical solution of the embodiment convenient to extend to different inertial motion capture systems, greatly improving the practicality, economy and adaptability of the inertial motion capture system.

[0045] In further embodiments, the calibration parameter solving network uses a loss function combining a calibration error loss function and a prior loss function for supervised learning, which improves the accuracy and robustness of the calibration parameter solving network in estimating calibration parameters.

[0046] Further, in the embodiment, the continuous Euler angle space is divided according to a preset angle, the Euler angle space is converted into a discrete space, and the rotation richness of the IMU rotation reading sequence is further calculated, so that the rotation richness can be quantified more accurately, and the reliability of the calibration parameter estimation can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 A flowchart of an IMU calibration method for an inertial motion capture system according to an embodiment of the present application;

[0048] Figure 2 An IMU dynamic calibration flowchart of an IMU calibration method for an inertial motion capture system according to an embodiment of the present application;

[0049] Figure 3 An IMU calibration device diagram for an inertial motion capture according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] To further illustrate the embodiments, the present application provides drawings. These drawings are part of the disclosure of the present application, mainly used to illustrate the embodiments, and can be used to explain the operating principle of the embodiments in conjunction with the related description of the specification. Those of ordinary skill in the art should understand other possible implementations and advantages of the present application in conjunction with these. The components in the drawings are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0051] The present application will be further described in conjunction with the drawings and specific embodiments.

[0052] Embodiment one:

[0053] In order to improve the accuracy of inertial motion capture and optimize the user experience, so that the inertial motion capture system remains efficient and stable in long-term continuous use, the present embodiment proposes an IMU calibration method for an inertial motion capture system. Figure 1 A flowchart of an IMU calibration method for an inertial motion capture system according to an embodiment of the present application. As shown in Figure 1 The inertial motion capture system includes a plurality of IMUs, a calibration parameter solving network, and a pose estimation model, and the method includes:

[0054] Step S1, the measurement data of the plurality of IMUs is obtained at a predetermined time interval, wherein the measurement data includes an IMU rotation reading sequence of a predetermined length;

[0055] Step S2, calculate the rotation richness of the IMU rotation reading sequence of each IMU, wherein the rotation richness of the IMU rotation reading sequence is: dividing the continuous Euler angle space according to a predetermined angle, and then converting it into a discrete space of a predetermined size, and counting the number of rotation readings in the IMU rotation reading sequence that are greater than 0 in the discrete space;

[0056] Step S3, input the IMU rotation reading sequence of each IMU into the calibration parameter solving network, obtain the estimated value of the calibration parameter of each IMU through the calibration parameter solving network, and update the calibration parameter of the IMU whose rotation richness is higher than a predetermined threshold using the estimated value of the calibration parameter, wherein the calibration parameter includes: a global coordinate system drift matrix and a wearing position offset matrix, and the calibration parameter solving network includes: an encoder and a Transformer pooling mapping (TPM) module;

[0057] Wherein, the calibration parameter solving network is modeled as follows:

[0058] f Glob (·)=TPM Glob (E(·))

[0059] f Loca (·)=TPM Loca (E(·))

[0060] f Glob represents a global coordinate system drift matrix solving function, f Loca represents a wearing position offset matrix solving function, TPM Glob represents a TPM module for solving the global coordinate system drift matrix, TPM Loca represents a TPM module for solving the wearing position offset matrix, and E represents an encoder.

[0061] Step S4, calculate the corresponding global joint pose according to the updated calibration parameter and the IMU rotation reading sequence corresponding thereto, and input the global joint pose into a pose estimation model;

[0062] Wherein, the global joint pose at the corresponding time t is calculated by the following formula:

[0063]

[0064] R IMU represents the first IMU rotation reading sequence, represents the global coordinate system drift matrix, represents the global joint pose, represents the wearing position offset matrix, and G is a predetermined ideal world coordinate system, and G* is a real IMU world coordinate system.

[0065] Embodiment Two

[0066] The IMU may have coordinate system drift and wearing position offset during use, causing dynamic changes in the calibration parameters. However, the existing method cannot update the calibration parameters during motion capture. Once the IMU drifts or the position changes, the user must re-wear the device and perform calibration again, causing the motion capture process to be interrupted. Therefore, the technical solution of the embodiment can update the calibration parameters in real time during motion capture, effectively solving the problem of accumulated attitude estimation errors caused by IMU drift and wearing position offset during long-term use, and without the need for the user to re-wear the device or maintain a specific calibration action, ensuring continuous user experience.

[0067] Figure 2 A dynamic calibration flowchart for the IMU calibration method for the inertial motion capture system according to an embodiment of the application. The technical solution of the embodiment uses a Transformer model to estimate the calibration parameters, which takes the real-time IMU rotation reading sequence collected by the inertial motion capture system as input and outputs the estimated value of the calibration parameters. At the same time, the technical solution uses a calibration parameter update strategy based on rotation diversity (RD), which only performs calibration when the rotation diversity is higher than a threshold, as shown in Figure 2 The method comprises the following steps:

[0068] Step 1: Obtain the latest rotation reading sequence of a predetermined length for each IMU according to a predetermined time interval; preferably, the time interval is 1s; preferably, the predetermined length is 128 frames; preferably, for a sparse layout inertial motion capture system, for example, using only 6 IMUs, the global acceleration measurement value measured by the IMU is also inputted;

[0069] Let the global joint attitude required by the inertial motion capture system be The global acceleration is a G The original IMU attitude and acceleration measurement value can be represented as:

[0070]

[0071]

[0072] wherein R IMU represents the IMU rotation reading sequence, a IMU represents the IMU acceleration measurement value, represents the global joint attitude, a G represents the global acceleration, represents the global coordinate system drift matrix, ​denoted as the wearer position offset matrix, t represents the corresponding time such as moment, G is the predetermined ideal world coordinate system, and G* is the real IMU world coordinate system.

[0073] The goal of IMU calibration is to accurately solve the two matrices mentioned above, thereby converting the original IMU rotation reading sequence into the input required by the inertial motion capture system. As shown in the formula above, the IMU rotation reading sequence is the calibration parameter... and And skeletal movement, such as The coupling, and the core objective of dynamic calibration in this embodiment is to construct two functions f Glob with f Loca This allows the two to separately and Separate from the raw IMU readings.

[0074] Step 2: Calculate the rotation richness of the IMU rotation reading sequence for each IMU;

[0075] The accurate solution of calibration parameters by the TIC network depends on the rotation richness of the input IMU rotation reading sequence. This is because the rotation readings of the IMU under different attitudes can provide richer solution constraints and reduce the solution space of the calibration parameters. Therefore, it is desirable to trigger calibration only when the rotation richness is sufficient.

[0076] In one specific implementation, to quantify rotational richness, the rotational richness (RD) of the IMU rotational reading sequence is calculated as shown in Table 1 below. An IMU rotational reading sequence R of predetermined length n is input. IMU (1)…R IMU (n), representing the continuous Euler angle space (θ) x ∈[-180, 180], θ y ∈[-90, 90], θ z The region ∈ [-180, 180] is divided into a 24×12×24 discrete space S by dividing it into predetermined angles, such as 15 degrees. Each element in the discrete space is initialized to 0. Then, the input IMU rotation reading sequence is traversed, and each reading R is... IMU (t) is mapped to the corresponding positions i, j, k in S, and the count of the corresponding positions is accumulated. Here, the value of t ranges from 1 to n. Finally, the rotation richness RD is obtained by calculating the number of elements in S that are greater than zero. This refers to the universal quantifier in logical operators. It represents the three-dimensional real number space.

[0077]

[0078] Table 1

[0079] Step 3, input the IMU rotation reading sequence of each IMU into the calibration parameter solving network, solve the estimated value of the calibration parameter of each IMU through the calibration parameter solving network, and update the calibration parameter of the IMU whose rotation richness is higher than the preset threshold using the estimated value of the calibration parameter;

[0080] In a specific implementation, the input data of the TIC network can be replaced by a combination of the IMU rotation reading sequence and other measurable data, such as acceleration, magnetometer reading, etc.

[0081] Preferably, to construct f Glob and f Loca , the embodiment technical scheme proposes a calibration parameter solving network TIC, which is implemented based on a Transformer neural network model and consists of two components:

[0082] 1) Encoder (E): a network consisting of 3 layers of Transformer encoder modules, used for time-space feature extraction;

[0083] 2) Transformer-Pooling-Mapping (TPM, Transformer Pooling Mapping) module: a module consisting of one layer of Transformer encoder, time average pooling, and one layer of linear mapping.

[0084] In a specific implementation, the TIC network can be replaced by other types of neural networks, such as convolutional neural networks, recurrent neural networks, etc.

[0085] In a specific implementation, the TIC network contains two TPM modules, TPM Glob and TPM Loca for solving the global coordinate system drift matrix and the worn position offset matrix, respectively.

[0086] Therefore, the TIC network models f Glob and f Loca as follows:

[0087] f Glob (·) = TPM Glob (E(·))

[0088] f Loca (·) = TPM Loca (E(·))

[0089] (·) represents input parameters of the same dimension;

[0090] In one specific implementation, for a TIC network suitable for a 6-IMU inertial motion capture system, the structural details are shown in Table 2 below:

[0091] Parameter name Value Input dimension 54 Number of encoder layers 3 Token dimension 256 Number of multi-head attentions 8 Feed-forward network hidden layer dimension 512 Output dimension 36+36

[0092] Table 2

[0093] The TIC network takes as input a sequence of 6-IMU rotation matrix format pose measurements, and outputs the calibration parameters for each of the 6-IMUs wherein, are all in R6D format.

[0094] Preferably, the technical solution of the present embodiment uses random samples dynamically generated based on a large-scale motion capture (AMASS) dataset for model training; in one specific implementation, first, fully calibrated IMU data is obtained from human motion data in the AMASS dataset which can simulate accurate skeletal poses; each training sample has a length of 128 frames of data at 30Hz, corresponding to a time length of 4.26 seconds; further, by adding a random global coordinate system drift matrix and a wearing position offset matrix to the are randomly sampled from a uniformly distributed Euler angle space, and the Euler angle space random sampling space is shown in Table 3 below:

[0095]

[0096] Table 3

[0097] Preferably, since the synthesized training data contains but the samples do not contain all possible cases, in actual use, the update of the calibration parameters will be in the form of change estimation, ensuring that the are always within the distribution range of the training data, so that the technical solution of the present embodiment can adapt to all possible sensor drifts and wearing position offsets.

[0098] Preferably, the TIC network is trained using supervised learning, and the loss function used is a loss function L all that combines a calibration error loss function L cali , defined as: ​​​​​​​​

[0099]

[0100] wherein, denotes the IMU rotation readings in the generated training data, denotes the global coordinate drift matrix corresponding to the generated training data, denotes the wearing position offset matrix corresponding to the generated training data, denotes the square of the L2 norm, n denotes the predetermined length of the IMU rotation reading sequence; the calibration error loss function will guide the neural network to learn how to separate the calibration parameters that can be regarded as fixed values in a short time from the original IMU readings;

[0101] In addition to the calibration error loss, the technical solution of the embodiment also adopts an additional prior loss function L prior for TIC network training, wherein:

[0102]

[0103] for the calibrated IMU rotation readings, I denotes the unit matrix; the prior loss function will guide the neural network to estimate the value of the calibration parameters as the unit matrix when the input data is completely calibrated, and provide further constraints for the training of the neural network;

[0104] The final loss function used is L cali in combination with L prior

[0105] L all = L cali + L prior

[0106] Preferably, the data batch size batchsize in the model training process is 256, the learning rate is 0.001, the Adam optimizer is used, and the training round epoch is 20.

[0107] Step 4, calculating according to the updated correction parameters and the IMU rotation reading sequence corresponding thereto to obtain the global joint posture, and inputting the global joint posture into the pose estimation model; in one specific implementation, it further includes: calculating according to the estimated value of the correction parameters and the acceleration measurement value to obtain the global acceleration, and inputting the global acceleration into the pose estimation model.

[0108] ​In one specific implementation, the dynamic IMU calibration of the present embodiment can be run synchronously with the inertial motion capture system based on the rotation richness calculation of the trained TIC network and the IMU rotation reading sequence. Table 4 below shows the specific algorithm of the inertial motion capture dynamic IMU calibration. Specifically, the input data of the algorithm is the uncalibrated IMU rotation reading R IMU (1), R IMU (2), and the acceleration reading a IMU(1) of the IMU IMU(2) , a n ,..., and the return value is the calibrated IMU data Initialize the data index i to 1 and create a data buffer B with a maximum length of n n , and initialize it with a unit matrix and Preferably, the maximum length n is the predetermined length of the IMU rotation reading sequence.

[0109] Each loop takes a new IMU rotation reading from the uncalibrated IMU rotation reading, which is passed through and calibration and added to B n ;

[0110] When the number of rotation readings added in B n reaches the maximum length, calculate the rotation richness RD of the rotation reading sequence in B

[0111] If RD is greater than the preset threshold T R , it is considered that there is enough rotation change to perform a calibration. At this time, call the TIC network to obtain the correction value and and update and

[0112] Then, empty B n , the data index is incremented, and the next round of data collection is prepared.

[0113]

[0114] Table 4

[0115] In summary, the IMU calibration currently used in inertial motion capture systems relies on fixed calibration parameters calculated through specific calibration motions at the start of the system, and cannot adapt to the dynamic changes of calibration parameters in the subsequent use process. The technical solution of the embodiment realizes real-time estimation and update of calibration parameters by using IMU readings in the inertial motion capture process, effectively guaranteeing the attitude estimation accuracy of the inertial motion capture system in the long-time use process. At the same time, the calibration parameter estimation model used in the technical solution of the embodiment can be trained using only synthetic data, which makes the solution convenient to extend to inertial motion capture systems with different IMU settings.

[0116] Embodiment three:

[0117] The application further provides an IMU calibration device for inertial motion capture, as shown in the figure, the device comprises a processor 301, a memory 302, a bus 303, and a computer program stored in the memory 302 and executable on the processor 301, the processor 301 comprises one or more processing cores, the memory 302 is connected to the processor 301 through the bus 303, the memory 302 is used for storing program instructions, and the processor executes the computer program to realize the steps in the above method embodiments of the embodiment one of the application. Figure 3

[0118] Further, as an executable solution, the IMU calibration device for inertial motion capture can be a computer unit, which can be a desktop computer, a notebook computer, a palm computer, a cloud server and the like. The computer unit can include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above-mentioned component structure of the computer unit is only an example of the computer unit, and does not constitute a limitation on the computer unit, and can include more or fewer components than the above, or combine certain components, or different components. For example, the computer unit can also include an input / output device, a network access device, a bus and the like, and the embodiments of the application do not limit this.

[0119] ​Further, as an implementation, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor or the like, which is a control center of the computer unit, and connects all parts of the computer unit through various interfaces and lines.

[0120] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the computer unit by running or executing the computer program and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function; and the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0121] Embodiment Four

[0122] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the steps of the method in the above embodiments of the application.

[0123] The modules / units integrated by the computer units, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can realize the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms, etc. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM) and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0124] Embodiment five:

[0125] The application also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the method as described above.

[0126] Although the application has been specifically shown and described with respect to the preferred embodiments, it should be understood that various changes in form and detail can be made therein without departing from the spirit and scope of the application as defined in the appended claims.

Claims

1. An IMU calibration method for an inertial motion capture system, characterized in that, The inertial motion capture system comprises a plurality of IMUs, a calibration parameter solving network, and a pre-constructed pose estimation model, comprising: Step S1, the measurement data of the plurality of IMUs is obtained at a predetermined time interval, wherein the measurement data comprises a sequence of IMU rotation readings of a predetermined length; Step S2, the rotation richness of the sequence of IMU rotation readings of each IMU is calculated, wherein the rotation richness of the sequence of IMU rotation readings is that a continuous Euler angle space is divided according to a predetermined angle, and then converted into a discrete space of a predetermined size, and the number of rotation readings greater than 0 in the sequence of IMU rotation readings in the discrete space is counted; Step S3, the sequence of IMU rotation readings of each IMU is input into the calibration parameter solving network, and the estimated value of the calibration parameter of each IMU is obtained by solving the calibration parameter solving network, and the calibration parameter of the IMU with a rotation richness higher than a predetermined threshold is updated using the estimated value of the calibration parameter, wherein the calibration parameter comprises a global coordinate system drift matrix and a wearing position offset matrix, and the calibration parameter solving network comprises an encoder and a Transformer pooling mapping (TPM) module; Wherein, the calibration parameter solving network is modeled as follows: f Glob (·) = TPM Glob (E(·)) f Loca (·) = TPM Loca (E(·)) f Glob represents a global coordinate frame drift matrix solving function, f Loca represents a worn position offset matrix solving function, TPM Glob represents a TPM module for solving the global coordinate frame drift matrix, TPM Loca represents a TPM module for solving the worn position offset matrix, E represents the encoder; Step S4, the corresponding global joint pose is obtained by calculating the updated calibration parameter and the corresponding sequence of IMU rotation readings, and the global joint pose is input into the pose estimation model; Wherein, the global joint pose at time t is calculated by the following formula: R IMU represents a sequence of first IMU rotation readings, represents the global coordinate frame drift matrix, represents the global joint poses, represents the worn position offset matrix, G is a predetermined ideal world coordinate frame, and G* is a real IMU world coordinate frame.

2. The IMU calibration method of claim 1, wherein, When the number of IMUs is greater than a predetermined number, the measurement data further comprises IMU acceleration measurement values; The step S4 further comprises calculating the global acceleration at time t using the following formula according to the IMU acceleration measurement values, and inputting the global acceleration into the pose estimation model, wherein: a IMU denotes the IMU acceleration measurements, a G denotes the global acceleration, denotes the global coordinate system drift matrix.

3. The IMU calibration method of claim 2, wherein, The predetermined number is 6.

4. The IMU calibration method of claim 1, wherein, The calibration parameter solving network is implemented based on a Transformer model, wherein the encoder is a network composed of multiple layers of Transformer encoder modules, used to extract the time-space features of the first sequence of IMU rotation readings; and the TPM module is composed of one layer of Transformer encoder, time average pooling, and one layer of linear mapping.

5. The IMU calibration method of claim 1, wherein, The calibration parameter solving network uses random samples dynamically generated from a predetermined large-scale motion capture dataset for model training, wherein: Fully calibrated IMU data is obtained from human motion data in the large-scale motion capture dataset, and random global coordinate system drift matrices and wearing position offset matrices are added to the fully calibrated IMU data to generate training data, wherein the fully calibrated IMU data can simulate accurate bone poses, and the global coordinate system drift matrices and wearing position offset matrices are randomly sampled from a pre-set uniform distribution of Euler angle space.

6. The IMU calibration method of claim 5, wherein, The calibration parameter solving network is trained in a supervised learning manner, and a loss function combining a correction error loss function and a prior loss function is used in the training of the supervised learning manner, wherein: The correction error loss function is: L cali represents the correction error loss function, represents the generated training data IMU rotation readings, represents the generated training data corresponding global coordinate drift matrix, represents the generated training data corresponding worn position offset matrix, represents the square of the L2 norm, n represents a predetermined length of the IMU rotation readings sequence; The prior loss function is: L prior represents the prior loss function, represents the calibrated IMU rotation readings, I represents an identity matrix, and n represents a predetermined length of the sequence of IMU rotation readings. The loss function combining the correction error loss function and the prior loss function is: L all = L cali + L prior L all represents a loss function used by the calibration parameter solving network.

7. An inertial motion capture system characterized by, The inertial motion capture system implements the IMU calibration method as claimed in any one of claims 1 to 6 in runtime, comprising: a plurality of IMUs configured to obtain IMU measurement data; a calibration parameter solving network configured to solve IMU rotation reading sequences in the obtained IMU measurement data; a pose estimation model configured to simulate a pose according to the obtained global joint pose and global acceleration.

8. An IMU calibration apparatus for inertial motion capture, characterized by, a memory and a processor, the memory storing at least one program, and the at least one program being executed by the processor to implement the IMU calibration method as claimed in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The storage medium stores at least one program, and the at least one program is executed by the processor to implement the IMU calibration method as claimed in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the IMU calibration method as claimed in any one of claims 1 to 6.

Images