Sparse IMU (Inertial Measurement Unit) motion capture method capable of resisting magnetic field interference
By combining the joint detection of acceleration, angular velocity and magnetic field data with neural network calibration in the IMU motion capture system, the problem of inaccurate attitude estimation under magnetic field interference in sparse IMU motion capture is solved, and higher precision motion capture effect is achieved.
Patent Information
- Application Number
- CN202511298938.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-11
- Publication Date
- 2025-11-18
AI Technical Summary
Existing IMU motion capture methods are susceptible to magnetic field interference under sparse configurations, resulting in a high false negative rate and affecting the continuity and accuracy of attitude estimation. Their effectiveness is particularly limited in complex electromagnetic environments.
By acquiring raw data from IMUs attached to the ends of the limbs, head, and waist, the rotation angle is initially estimated using acceleration and angular velocity information. This is then combined with magnetic field interference detector data and magnetic field data from neighboring IMUs for joint judgment. Finally, a neural network model based on prior knowledge of human motion is used to calibrate the rotation error, further improving the estimation accuracy.
More accurate pose estimation and global motion estimation were achieved in environments with magnetic field interference, which expanded the stability and accuracy of sparse IMU motion capture systems in everyday environments.
Smart Images
Figure CN120970633A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of IMU motion capture, and particularly relates to a sparse IMU motion capture method resisting magnetic field interference. BACKGROUND
[0002] As an important sensing component of motion capture systems, inertial measurement units (IMUs) are widely used in virtual reality, film production, medical rehabilitation, human-computer interaction and other fields. In the related art, through the collaborative work of accelerometers, angular rate sensors and magnetometers, an attitude estimation algorithm system based on sensor fusion is constructed. Specifically, this technology covers the whole process from raw data acquisition, local attitude estimation to global motion reconstruction, including Kalman filtering, error state filtering, neural network modeling and other key links. Among them, the magnetic field information plays a key role in the north calibration of attitude estimation, but its characteristics of being easily affected by external electromagnetic interference become an important bottleneck for system stability and accuracy improvement.
[0003] However, in the existing IMU rotation estimation algorithm, a "detection-rejection" mechanism is usually used to handle magnetic field interference, relying only on a single indicator such as magnetic field length or magnetic inclination for judgment, without fully considering the collaborative relationship between the spatial distribution characteristics of multiple IMUs and the human motion prior in the motion capture scene, which may result in a high rate of missed detection, and the interference information is not effectively excluded, thereby affecting the continuity and accuracy of attitude estimation. Especially in the case of sparse IMU configuration, local errors are difficult to be effectively corrected by traditional methods, further limiting the application effect of the system in complex electromagnetic environments. SUMMARY
[0004] The present application aims to at least partially solve one of the technical problems in the related art.
[0005] To this end, a first object of the present application is to provide a sparse IMU motion capture method resisting magnetic field interference.
[0006] A second object of the present application is to provide a sparse IMU motion capture device resisting magnetic field interference.
[0007] A third object of the present application is to provide an electronic device.
[0008] A fourth object of the present application is to provide a computer-readable storage medium.
[0009] A fifth object of the present application is to provide a computer program product.
[0010] To achieve the above objects, the first aspect of the present application provides a sparse IMU motion capture method resisting magnetic field interference, comprising:
[0011] Obtaining raw data of at least six IMUs bound to the extremities of the limbs, the head and the waist of the human body, the raw data including acceleration, angular velocity and magnetic field information;
[0012] Using the acceleration and angular velocity information, the rotation angle of the IMU is preliminarily estimated;
[0013] Based on the rotation angle, the magnetic field data of each IMU is detected by a magnetic field interference detector, and the posture estimation at the previous moment and the magnetic field data of the adjacent IMU are combined to jointly determine whether the current IMU is interfered by the magnetic field; if no magnetic field interference is detected, the rotation of the IMU is calibrated using the current magnetic field data;
[0014] The calibrated IMU rotation data and acceleration information are input into a neural network model based on human motion prior, the rotation error of the IMU is calibrated, and it is converted into a rotation matrix, and the rotation and acceleration of the five IMUs are further calibrated;
[0015] The calibrated IMU rotation and acceleration are input into an existing sparse IMU motion capture algorithm to estimate the overall motion of the user.
[0016] Optionally, it also includes: reading the raw data of each IMU in real time at a sampling frequency of not less than 100Hz.
[0017] Optionally, based on the rotation angle, the magnetic field data of each IMU is detected by a magnetic field interference detector, and the posture estimation at the previous moment and the magnetic field data of the adjacent IMU are combined to jointly determine whether the current IMU is interfered by the magnetic field, including:
[0018] Using an existing magnetic field interference detector to individually detect the magnetic field data of each IMU;
[0019] The relative position of the IMU is calculated through the human body posture estimated at the previous moment;
[0020] The magnetic field interference detection results of the current IMU and its adjacent two IMUs are aggregated to jointly determine the magnetic field interference, and if no magnetic field interference is detected by the three IMUs, it is considered that the space where the current IMU is located has no magnetic field interference.
[0021] Optionally, the calibrated IMU rotation data and acceleration information are input into a neural network model based on human motion prior, the rotation error of the IMU is calibrated, and it is converted into a rotation matrix, and the rotation and acceleration of the five IMUs are further calibrated, including:
[0022] By using the existing IMU rotation data, the acceleration measured by each IMU is converted to the global coordinate system;
[0023] The acceleration and rotation information of the six IMUs is input into an LSTM neural network, and the rotation errors of the five IMUs except the root node are estimated by learning the human motion prior;
[0024] The estimated rotation errors are converted into rotation matrices, and the IMU rotation and acceleration are further calibrated.
[0025] Optionally, the training phase of the LSTM neural network includes:
[0026] Motion data is extracted from a human motion dataset, and the positions and rotations of virtual IMUs are calculated and denoted as R_gt;
[0027] The disturbed magnetic field data is synthesized by a simulation method, and the rotation data R_noise is generated by the rotation estimation algorithm of the IMU, and the erroneous global acceleration a_noise is derived from the rotation data R_noise;
[0028] The R_noise and a_noise of the six IMUs are taken as inputs, the rotation error between R_gt and R_noise is taken as supervision, and the rotation error is optimized by an L2 loss function, so that the trained network can adapt to the actual magnetic field interference.
[0029] To achieve the above purpose, the second aspect embodiment of the present application proposes an anti-magnetic interference sparse IMU motion capture device, comprising:
[0030] An IMU data acquisition module is configured to acquire original data of at least six IMUs bound to the ends of human limbs, the head and the waist, wherein the original data includes acceleration, angular velocity and magnetic field information;
[0031] A rotation state estimation module is configured to preliminarily estimate the rotation angle of the IMU by using the acceleration and angular velocity information;
[0032] A magnetic field interference judgment and calibration module is configured to detect the magnetic field data of each IMU by a magnetic field interference detector based on the rotation angle, and jointly judge whether the current IMU is interfered by the magnetic field in combination with the posture estimation at the previous moment and the magnetic field data of the adjacent IMU; if no magnetic field interference is detected, the rotation of the IMU is calibrated using the current magnetic field data;
[0033] A neural network error estimation and calibration module is configured to input the calibrated IMU rotation data and acceleration information into a neural network model based on human motion prior, calibrate the rotation error of the IMU, and convert it into a rotation matrix to further calibrate the rotation and acceleration of the five IMUs;
[0034] an output module configured to input the calibrated IMU rotation and acceleration into an existing sparse IMU motion capture algorithm to estimate the overall motion of the user.
[0035] To achieve the above object, the third aspect of the present application provides an electronic device, comprising: a processor, and a memory connected with the processor in communication;
[0036] The memory stores computer-executable instructions.
[0037] The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of the first aspect.
[0038] To achieve the above object, the fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method according to any one of the first aspect.
[0039] To achieve the above object, the fifth aspect of the present application provides a computer program product, wherein the computer program is executed by a processor to implement the method according to any one of the first aspect.
[0040] The technical scheme provided by the embodiments of the present application at least brings the following beneficial effects: more accurate pose estimation and global motion estimation can be obtained in an environment with magnetic field interference. This is conducive to the expansion of the application scenarios of the sparse IMU motion capture system, and the system can stably operate in daily environments with magnetic field interference (such as supermarkets, classrooms, office buildings, etc.).
[0041] Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and / or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS
[0042] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:
[0043] Figure 1 A flowchart of a sparse IMU motion capture method against magnetic field interference provided by an embodiment of the present application;
[0044] Figure 2 A flowchart of a sparse IMU motion capture method against magnetic field interference provided by an embodiment of the present application;
[0045] Figure 3 A structural schematic diagram of a sparse IMU motion capture device against magnetic field interference provided by an embodiment of the present application. DETAILED DESCRIPTION
[0046] Embodiments of the present application are described in detail below with reference to the attached drawing figures, wherein the same or like component have the same or similar designations. The embodiments described below are presented by way of example only and are not intended to limit the present application as defined by the appended claims.
[0047] Figure 1 and Figure 2 A flowchart of a method for anti-magnetic interference sparse IMU motion capture according to an embodiment of the present application is shown in FIG. 1. The method comprises the following steps: Figure 1 and Figure 2
[0048] S1, obtaining raw data of at least six IMUs attached to the ends of the limbs, head and waist of the human body, the raw data including acceleration, angular velocity and magnetic field information.
[0049] In the embodiments of the present application, the first step is to obtain raw data of at least six IMUs attached to the ends of the limbs, head and waist of the human body. Specifically, the user needs to install six IMUs on different parts of the human body, including the ends of the limbs, the head and the waist. The sensors of each IMU will measure and collect acceleration, angular velocity and magnetic field data in real time. These raw data are the basis for subsequent processing in this method and can provide important information for rotation estimation and motion capture. In order to ensure the accuracy and real-time performance of the data, the sampling frequency of the IMU is at least 100 Hz, i.e. at least 100 data readings per second, ensuring that the system can capture the dynamic changes of the human body at a high frequency.
[0050] The acceleration, angular velocity and magnetic field information measured by these sensors will be used for subsequent rotation estimation and interference detection processing. The acceleration information provides the acceleration change of the IMU in space, the angular velocity information helps to estimate the rotation speed of the IMU, and the magnetic field information helps to calibrate and determine the posture and direction of the IMU. After obtaining these raw data, the system will continue data fusion and interference detection to achieve accurate motion capture and human motion analysis.
[0051] S2, preliminarily estimating the rotation angle of the IMU using acceleration and angular velocity information.
[0052] In the embodiments of the present application, the second step is to preliminarily estimate the rotation angle of the IMU using acceleration and angular velocity information. Existing IMU rotation estimation algorithms, such as Kalman filter, extended Kalman filter, error state Kalman filter, etc., usually use magnetic field information to calibrate the rotation angle selectively according to actual needs. However, in the present application, we first rely on the acceleration and angular velocity information measured by the IMU to preliminarily estimate the rotation angle.
[0053] Specifically, the acceleration information reflects the acceleration changes of the IMU in space, and the tilt angle of the IMU is usually calculated by measuring the gravitational acceleration. The angular velocity information directly provides the rate of rotation of the IMU, and by integrating the angular velocity, the rotation angle of the IMU at a certain time point can be obtained. By combining these two kinds of information, a preliminary rotation angle estimation can be made without relying on magnetic field data. At this time, the acceleration and angular velocity information provide a dynamic description of the spatial position and motion of the IMU, thereby laying a foundation for subsequent rotation estimation and calibration.
[0054] The purpose of this step is to obtain the preliminary rotation angle of the IMU as the input for subsequent magnetic field interference judgment and calibration, ensuring that the system can make effective preliminary estimation even in the absence of reliable magnetic field data. By combining acceleration and angular velocity information, we can minimize the impact of magnetic field interference as much as possible and provide a preliminary reference for subsequent accurate estimation.
[0055] S3, based on the rotation angle, the magnetic field data of each IMU is detected by the magnetic field interference detector, and the posture estimation at the previous time and the magnetic field data of the adjacent IMU are combined to jointly judge whether the current IMU is interfered by the magnetic field; if no magnetic field interference is detected, the rotation of the IMU is calibrated using the current magnetic field data.
[0056] In the embodiments of the present application, the third step is to detect the magnetic field data of each IMU by the magnetic field interference detector based on the rotation angle, and combine the posture estimation at the previous time and the magnetic field data of the adjacent IMU to jointly judge whether the current IMU is interfered by the magnetic field; if no magnetic field interference is detected, the rotation of the IMU is calibrated using the current magnetic field data.
[0057] First, existing IMU rotation estimation algorithms usually have built-in magnetic field interference detectors. The present application uses these existing magnetic field detectors to detect the magnetic field of each IMU individually. These detectors can identify whether there is an anomaly or interference by judging the change of the magnetic field data, such as sudden changes in the magnetic field length or inclination angle. However, relying solely on the magnetic field detection of each IMU may miss some cases, especially when multiple IMUs are working simultaneously and there is local interference. To solve this problem, the present application adopts a joint judgment method.
[0058] In this process, the application combines the human posture estimated at the previous moment to calculate the position of the IMU in space. At the same time, the application also aggregates and analyzes the magnetic field detection results of each IMU with the magnetic field data of the two IMUs closest to it. The combination of the magnetic field data of the three can provide more comprehensive information. Only when none of the three IMUs detects magnetic field interference, the application considers that there is no magnetic field interference in the space where the current IMU is located. This method helps to avoid missing interference caused by individual IMU misjudgment, ensuring more accurate judgment of magnetic field interference. If no magnetic field interference is found during detection, the application will use the magnetic field data of the current IMU to calibrate the rotation angle estimated in the previous step. Magnetic field data can provide directional information of the IMU, and then accurately adjust the rotation angle. The specific magnetic field calibration steps depend on existing IMU rotation estimation algorithms, which can effectively calibrate the rotation estimation results of the IMU by combining acceleration, angular velocity and magnetic field data.
[0059] Through this combined detection and calibration method, the application can effectively improve the accuracy of IMU rotation estimation in a magnetic field interference environment, avoid the missed detection and misjudgment problems that may occur in individual IMU detection, and ensure the stability and reliability of the system.
[0060] S4, input the calibrated IMU rotation data and acceleration information into the neural network model based on human motion prior, calibrate the rotation error of the IMU, and convert it into a rotation matrix to further calibrate the rotation and acceleration of the five IMUs.
[0061] In the embodiment of the application, the fourth step is to input the calibrated IMU rotation data and acceleration information into the neural network model based on human motion prior, calibrate the rotation error of the IMU, and convert it into a rotation matrix to further calibrate the rotation and acceleration of the five IMUs.
[0062] In the previous step, the application effectively alleviates the impact of magnetic field interference through detection and aggregation, but due to missed detection or long-term magnetic field calibration, the rotation angle may still have errors. Therefore, in order to further improve the accuracy, the application introduces a neural network model based on human motion prior to calibrate the rotation error.
[0063] Firstly, the local acceleration data measured by IMUs are converted to global coordinate system by the estimated rotation angles. This conversion helps to integrate the acceleration data from different IMUs and ensures that the global motion information is considered in the subsequent rotation calibration. Then, the acceleration data of the 6 IMUs are input into the LSTM network together with the rotation data. LSTM (Long Short-Term Memory) is a neural network structure suitable for processing time series data, which can effectively model the time series relationship in human motion and further calibrate the rotation error using the motion prior of human body. It is worth noting that in this step, the rotation error of the root node (usually the torso) is not estimated, because the prior knowledge of human motion does not include the orientation information of the whole body - that is, whether the motion of the human body conforms to the joint constraints has nothing to do with whether the human body faces south or north, so the rotation of the root node cannot be involved in the calibration of the global rotation angle.
[0064] Once the neural network estimates the rotation error angle, the application converts it into a rotation matrix and uses it to further calibrate the rotation data and acceleration data of the 5 non-root node IMUs. This process can effectively eliminate errors introduced by magnetic field interference or sensor noise, and ultimately obtain more accurate IMU data.
[0065] Next, the training process of the neural network is described. Before being put into use, the LSTM network needs to be trained. In the training phase, the application extracts motion data from the human motion dataset and sets virtual IMUs on the relevant joints to calculate their accurate positions and rotations, which are denoted as R_gt. Then, the disturbed magnetic field is generated by magnetic field simulation, and the disturbed magnetic field data is synthesized according to the position and rotation information of the virtual IMUs. The acceleration and angular velocity information is synthesized by difference method, and the disturbed magnetic field measurement is obtained by querying the simulated magnetic field. Next, the IMU rotation estimation algorithm is used to estimate the rotation of the IMU from the synthesized acceleration, angular velocity and magnetic field data. Since the application has applied magnetic field interference, the estimated rotation angle (denoted as R_noise) will contain noise, and the global acceleration (denoted as a_noise) derived therefrom will also be affected. In the training process, the application takes R_noise and a_noise of the 6 IMUs as input, and the rotation error between R_gt and R_noise as a supervision signal. The training uses L2 loss function, the purpose is to minimize the predicted rotation error. Through this data synthesis method, the application can simulate the actual performance of the IMU under magnetic field interference from the bottom up, and can better generalize in actual application.
[0066] It should be noted that in order to improve the running efficiency of the system, the LSTM network is selected as the structure of the neural network in the present application, but the same technology can also be applied to other neural network architectures, such as the Transformer network, to process more complex time series data and capture more details of human motion.
[0067] Through this step, the accuracy of the IMU rotation is significantly improved by combining human motion prior information, and more accurate motion capture data can be provided in an environment with magnetic field interference.
[0068] S5, input the calibrated IMU rotation and acceleration into the existing sparse IMU motion capture algorithm to estimate the overall motion of the user.
[0069] In the embodiments of the present application, the fifth step is to input the calibrated IMU rotation and acceleration into the existing sparse IMU motion capture algorithm to estimate the overall motion of the user.
[0070] In the previous steps, the present application has calibrated the rotation and acceleration data of the IMU through multiple steps to ensure that these data have high accuracy, especially in a magnetic field interference environment. On this basis, the present application inputs the calibrated rotation data and acceleration information of the six IMUs as inputs into the existing sparse IMU motion capture algorithm.
[0071] The sparse IMU motion capture algorithm estimates the motion trajectory and posture of the human body by fusing the data of multiple IMUs. Generally, the sparse IMU motion capture algorithm can recover the full-body motion from a small amount of sensor data, and is particularly suitable for application scenarios that require convenient wearing or limited motion. In the present application, the six IMUs are installed at the ends of the limbs, the head and the waist of the human body, and these data provide sufficient information to estimate the overall motion of the human body. By inputting the rotation and acceleration data of each IMU into the sparse IMU motion capture algorithm, the system can perform data fusion through the algorithm model to estimate the motion of the human body, including the rotation and displacement of each part. This process involves processing the spatial relationship and time sequence of the IMU data, and through collaborative calculation of the IMU data, the algorithm can calculate a complete motion trajectory.
[0072] Finally, through this process, the present application can obtain the overall motion of the user, including the posture change and dynamic action trajectory. This estimation result not only reflects the motion state of each part where the IMU is located, but also obtains a comprehensive and accurate overall motion estimation result through the integration of the full-body data, thereby providing accurate data support for subsequent motion capture, motion analysis and virtual reality (VR), augmented reality (AR) and other applications.
[0073] In order to achieve the above-mentioned embodiments, the application further provides a sparse IMU motion capture device capable of resisting magnetic field interference. Figure 3 A sparse IMU motion capture device capable of resisting magnetic field interference is provided for the embodiments of the application. As shown in the structural schematic diagram of the device, Figure 3 The device comprises:
[0074] An IMU data acquisition module 100 is configured to acquire raw data of at least six IMUs bound to the ends of limbs, the head and the waist of a human body, wherein the raw data comprises acceleration, angular velocity and magnetic field information;
[0075] A rotation state estimation module 200 is configured to preliminarily estimate the rotation angle of the IMU by using the acceleration and angular velocity information;
[0076] A magnetic field interference judgment and calibration module 300 is configured to perform interference detection on the magnetic field data of each IMU by a magnetic field interference detector based on the rotation angle, and jointly judge whether the current IMU is interfered by the magnetic field in combination with the posture estimation at the previous moment and the magnetic field data of the adjacent IMU; if no magnetic field interference is detected, the rotation of the IMU is calibrated by using the current magnetic field data;
[0077] A neural network error estimation and calibration module 400 is configured to input the calibrated IMU rotation data and acceleration information into a neural network model based on human motion prior, calibrate the rotation error of the IMU, and convert it into a rotation matrix to further calibrate the rotation and acceleration of the five IMUs;
[0078] An output module 500 is configured to input the calibrated IMU rotation and acceleration into an existing sparse IMU motion capture algorithm to estimate the overall motion of the user.
[0079] As to the device in the above-mentioned embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.
[0080] In order to achieve the above-mentioned embodiments, the application further provides an electronic device, comprising a processor and a memory connected with the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to realize the method provided in the foregoing embodiments.
[0081] In order to achieve the above-mentioned embodiments, the application further provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to realize the method provided in the foregoing embodiments.
[0082] To achieve the above-mentioned embodiments, the application further provides a computer program product comprising a computer program which, when executed by a processor, implements the method provided by the above-mentioned embodiments.
[0083] The collection, storage, use, processing, transmission, provision and disclosure of the user's personal information involved in the present application comply with the relevant legal regulations and do not violate public order and good customs.
[0084] It should be noted that the personal information from the user should be collected for legal and reasonable purposes, and should not be shared or sold outside these legal uses. In addition, such collection / sharing should be carried out after the user's informed consent is received, including but not limited to informing the user to read the user agreement / user notice before the user uses the function, and signing the agreement / authorization including authorization of relevant user information. In addition, any necessary steps should be taken to protect and secure access to such personal information data, and to ensure that other people with access to personal information data comply with their privacy policies and processes.
[0085] The present application is expected to provide embodiments in which the user can selectively prevent the use or access of personal information data. That is, the present disclosure is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, the risk is minimized by limiting data collection and deleting data. In addition, such personal information is de-identified, if applicable, to protect the privacy of the user.
[0086] In the foregoing embodiment description, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples, without contradiction.
[0087] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited.
[0088] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.
[0089] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0090] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0091] Those skilled in the art can understand that all or part of the steps of the method of the above-mentioned embodiments can be instructed by a program to complete the relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, it includes one of the steps of the method embodiment or a combination thereof.
[0092] In addition, each functional unit in each embodiment of the present application can be integrated into one processing module, or each unit can exist physically independently, or two or more units can be integrated into one module. The integrated module can be realized in the form of hardware or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium.
[0093] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application. Those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.
[0094] It should be understood that various forms of flow shown above can be used to reorder, add or delete steps. For example, each step described in the present application can be executed in parallel, sequentially or in different order, as long as the desired results of the technical solutions of the present application can be achieved, which is not limited herein.
[0095] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and replacements can be made according to design requirements and other factors. Any modification, equivalent replacement and improvement within the spirit and principles of the present application shall be included in the scope of protection of the present application.
Claims
1. A sparse IMU motion capture method resistant to magnetic field interference, characterized in that, include: Acquire raw data from at least six IMUs attached to the extremities, head, and waist of the human body, including acceleration, angular velocity, and magnetic field information; Using acceleration and angular velocity information, the rotation angle of the IMU is initially estimated; Based on the rotation angle, the magnetic field data of each IMU is detected by a magnetic field interference detector. Combined with the attitude estimation of the previous moment and the magnetic field data of neighboring IMUs, it is jointly determined whether the current IMU is affected by magnetic field interference. If no magnetic field interference is detected, the rotation of the IMU is calibrated using the current magnetic field data; The calibrated IMU rotation data and acceleration information are input into a neural network model based on human motion priors to calibrate the rotation error of the IMU and convert it into a rotation matrix for further calibration of the rotation and acceleration of the five IMUs. The calibrated IMU rotation and acceleration are input into the existing sparse IMU motion capture algorithm to estimate the user's overall motion.
2. The method according to claim 1, characterized in that, Also includes: The raw data from each IMU is read in real time at a sampling frequency of not less than 100Hz.
3. The method according to claim 2, characterized in that, The method involves detecting magnetic field interference in the magnetic field data of each IMU based on the rotation angle using a magnetic field interference detector, and combining this with the attitude estimation from the previous moment and the magnetic field data of neighboring IMUs to jointly determine whether the current IMU is experiencing magnetic field interference. This includes: The magnetic field data of each IMU was detected individually using an existing magnetic field interference detector; The relative position of the IMU is calculated based on the human posture estimated in the previous moment; The magnetic field interference detection results of the current IMU and its two neighboring IMUs are aggregated to jointly determine the magnetic field interference situation. If no magnetic field interference is detected by any of the three IMUs, it is considered that there is no magnetic field interference in the space where the current IMU is located.
4. The method according to claim 3, characterized in that, The process involves inputting the calibrated IMU rotation data and acceleration information into a neural network model based on human motion priors to calibrate the IMU rotation error and convert it into a rotation matrix. Further calibration of the rotation and acceleration of the five IMUs is then performed, including: Using existing IMU rotation data, the acceleration measured by each IMU is transformed into the global coordinate system; The acceleration and rotation information of the six IMUs are input into the LSTM neural network, and the rotation error of the five IMUs other than the root node is estimated by learning human motion priors. The estimated rotation error is converted into a rotation matrix, and the rotation and acceleration of the IMU are further calibrated.
5. The method according to claim 4, characterized in that, The training phase of an LSTM neural network includes: Motion data is extracted from the human motion dataset, and the position and rotation of the virtual IMU are calculated, denoted as R_gt; Disturbed magnetic field data is synthesized using simulation methods, and rotational data R_noise is generated using the rotational estimation algorithm of the IMU. The erroneous global acceleration a_noise is then derived from the rotational data R_noise. Using the R_noise and a_noise of 6 IMUs as inputs, and the rotation errors of R_gt and R_noise as supervision, the rotation error is optimized through the L2 loss function, so that the trained network can adapt to the actual magnetic field interference.
6. A sparse IMU motion capture device resistant to magnetic field interference, characterized in that, include: The IMU data acquisition module is used to acquire raw data from at least six IMUs attached to the ends of the human limbs, head, and waist. The raw data includes acceleration, angular velocity, and magnetic field information. The rotation state prediction module is used to make a preliminary estimate of the rotation angle of the IMU using acceleration and angular velocity information; The magnetic field interference judgment and calibration module is used to detect interference in the magnetic field data of each IMU based on the rotation angle through the magnetic field interference detector, and combine the attitude estimation of the previous moment and the magnetic field data of neighboring IMUs to jointly determine whether the current IMU is affected by magnetic field interference. If no magnetic field interference is detected, the rotation of the IMU is calibrated using the current magnetic field data; The neural network error estimation and calibration module is used to input the calibrated IMU rotation data and acceleration information into the neural network model based on human motion priors, calibrate the rotation error of the IMU, and convert it into a rotation matrix to further calibrate the rotation and acceleration of the five IMUs. The output module is used to input the calibrated IMU rotation and acceleration into the existing sparse IMU motion capture algorithm to estimate the user's overall motion.
7. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-5.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-5.
9. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-5.