An inertial measurement unit based full-wireless human motion capture method and system

Abstract

The application discloses a kind of inertial measurement unit-based full-wireless human motion capture method and system, method includes: executing multi-node clock synchronization correction, obtain time-synchronized multi-node motion data;Respectively execute attitude solution to each sensing node, generate attitude estimate, and with accelerometer and magnetometer data as observation, recursively correct attitude estimate using extended Kalman filter;To the extended Kalman filter attitude estimate, execute adaptive complementary filtering secondary correction, acceleration credibility and magnetic field credibility are independently adjusted the correction weight of roll / pitch and heading;According to the final attitude estimate of each sensing node, with pelvic node as root node along tree-like kinematic chain, execute forward kinematics reconstruction, generate whole body motion attitude data. Realize that heading angle is high-precision stable estimation and multi-node accurate clock synchronization under complex environment, improve the precision and robustness of attitude estimation under dynamic motion condition.

Description

Technical Field

[0001] This invention belongs to the field of human motion capture technology, and particularly relates to a fully wireless human motion capture method and system based on an inertial measurement unit. Background Technology

[0002] Currently, mainstream motion capture technologies include two types: optical motion capture and inertial motion capture. Optical motion capture technology constructs a multi-view shooting environment using multiple high-speed cameras. It utilizes the two-dimensional image coordinates of markers placed on the human body surface and calculates the spatial position of the markers using a three-dimensional reconstruction algorithm. Typical products include the Vicon system and the OptiTrack system. Optical motion capture systems offer high positioning accuracy, but they suffer from drawbacks such as high system setup costs, strict limitations on ambient lighting and shooting space, marker occlusion issues, and difficulty in outdoor or complex environments.

[0003] Inertial motion capture technology uses inertial measurement units (IMUs) worn on major parts of the human body. Gyroscopes measure angular velocity, accelerometers measure linear acceleration, and magnetometers measure the direction of the Earth's magnetic field. Attitude calculation algorithms then calculate the three-dimensional attitude of each sensor node in real time, and finally reconstruct the whole-body motion using a human kinematic model. Typical products include the Xsens MVN system, the Perception Neuron series, and the Rokoko SmartSuit. Inertial motion capture is not limited by lighting conditions or spatial range, does not suffer from marker occlusion, and has relatively low system cost.

[0004] However, existing inertial motion capture systems still face the following technical bottlenecks in practical engineering applications:

[0005] (1) Heading angle drift problem. In indoor environments with strong magnetic interference (such as near reinforced concrete buildings and power facilities), the magnetometer data is severely affected by the distortion of the environmental magnetic field, resulting in significant drift in the heading angle estimation.

[0006] (2) Insufficient clock synchronization accuracy of multiple nodes. Each wireless sensor node uses an independent crystal oscillator. After long-term operation, clock drift accumulates, causing the data of multiple nodes to be unable to be accurately aligned on the time axis, which affects the accuracy of joint relative rotation calculation.

[0007] (3) Motion acceleration interference problem in attitude estimation. When the human body moves rapidly or violently, the resultant acceleration measured by the accelerometer includes motion acceleration components and gravitational acceleration components. Existing technology has difficulty in effectively distinguishing between the two, resulting in attitude estimation being affected by motion acceleration interference.

[0008] (4) Lack of standardized wearable solutions and exoskeleton device interface support.

[0009] Therefore, there is an urgent need for a fully wireless human motion capture method and system that can maintain stable heading angle estimation in complex environments, has high-precision multi-node clock synchronization, and can adaptively handle motion acceleration interference. Summary of the Invention

[0010] The purpose of this invention is to provide a fully wireless human motion capture method and system based on an inertial measurement unit, aiming to solve the technical problems existing in the prior art, such as heading angle drift, insufficient multi-node clock synchronization accuracy, motion acceleration interference with attitude estimation, and lack of standardized wearable solutions.

[0011] In a first aspect, the present invention provides a fully wireless human motion capture method based on an inertial measurement unit, comprising:

[0012] Acquire human motion data collected by multiple wireless inertial sensor nodes;

[0013] Perform multi-node clock synchronization correction on the human motion data to obtain time-synchronized multi-node motion data;

[0014] Attitude calculation is performed on the motion data of each sensor node in the multi-node motion data after time synchronization to generate attitude estimation values ​​for each sensor node. The current accelerometer data and magnetometer data are used as observations, and the attitude estimation values ​​are recursively corrected by extended Kalman filtering to output the extended Kalman filter attitude estimation values ​​for each sensor node.

[0015] An adaptive complementary filter secondary correction is performed on the extended Kalman filter attitude estimate to obtain the final attitude estimate of each sensor node.

[0016] Based on the final posture estimates of each sensor node, and combined with the parent-child hierarchical relationship between each limb segment in the human kinematics model, positive kinematic reconstruction is performed along the tree kinematic chain with the pelvic node as the root node to generate whole-body motion posture data.

[0017] Secondly, the present invention provides a fully wireless human motion capture system based on an inertial measurement unit, comprising:

[0018] The acquisition module is configured to acquire human motion data collected by multiple wireless inertial sensor nodes;

[0019] The first correction module is configured to perform multi-node clock synchronization correction on the human motion data to obtain time-synchronized multi-node motion data.

[0020] The calculation module is configured to perform attitude calculation on the motion data of each sensor node in the multi-node motion data after time synchronization, generate attitude estimation values ​​for each sensor node, and use the accelerometer data and magnetometer data at the current moment as observations, and recursively correct the attitude estimation values ​​using extended Kalman filtering, and output extended Kalman filter attitude estimation values ​​for each sensor node.

[0021] The second correction module is configured to perform adaptive complementary filtering secondary correction on the extended Kalman filter attitude estimate to obtain the final attitude estimate of each sensing node.

[0022] The generation module is configured to perform forward kinematic reconstruction along the tree kinematic chain with the pelvic node as the root node, based on the final posture estimates of each sensor node and the parent-child hierarchical relationship between each limb segment in the human kinematic model, to generate whole-body motion posture data.

[0023] Thirdly, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the fully wireless human motion capture method based on an inertial measurement unit according to any embodiment of the present invention.

[0024] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the program instructions are executed by a processor, the processor performs the steps of the fully wireless human motion capture method based on an inertial measurement unit according to any embodiment of the present invention.

[0025] The fully wireless human motion capture method and system based on inertial measurement units presented in this application have the following advantages:

[0026] To address the heading angle drift problem under magnetic interference environments, this invention employs a dual-layer fusion architecture combining extended Kalman filtering and adaptive complementary filtering, along with a magnetic interference hysteresis determination mechanism, to effectively suppress heading angle drift and sudden jumps under strong magnetic interference environments.

[0027] To address the issue of insufficient clock synchronization accuracy across multiple nodes, this invention employs a bidirectional clock synchronization message mechanism. By estimating the clock deviation and drift rate of each sensing node, the timestamps of each node are corrected, ensuring that the data of each node are aligned on a unified time axis, thus providing an accurate time reference for joint relative rotation calculations.

[0028] To address the problem of attitude estimation caused by motion acceleration, this invention determines the motion state by using the variance of the acceleration modulus. Under high acceleration motion, a correction intensity attenuation coefficient is generated based on the deviation between the acceleration modulus and gravitational acceleration, thereby achieving continuous adaptive adjustment of the correction intensity in the gravitational direction. This effectively avoids misjudging motion acceleration as attitude change and improves the accuracy of attitude estimation under dynamic motion conditions.

[0029] To address the issue of system scalability, this invention provides a standardized node human sensor layout scheme and reserves an interface for recognizing exoskeleton motion intentions, enabling the motion capture system to expand from simple data output to a real-time sensing front-end that can directly serve human-machine collaborative control. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 A flowchart illustrating a fully wireless human motion capture method based on an inertial measurement unit, as provided in an embodiment of the present invention;

[0032] Figure 2 This is a structural block diagram of a fully wireless human motion capture system based on an inertial measurement unit, provided in an embodiment of the present invention.

[0033] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Please see Figure 1 The diagram shows a flowchart of a fully wireless human motion capture method based on an inertial measurement unit according to this application.

[0036] like Figure 1 As shown, the fully wireless human motion capture method based on inertial measurement units specifically includes the following steps:

[0037] Step S101: Acquire human motion data collected by multiple wireless inertial sensor nodes.

[0038] In this step, the human motion data includes triaxial acceleration data, triaxial angular velocity data, and triaxial magnetic field strength data synchronously collected by each sensor node;

[0039] The wireless inertial sensing node comprises a nine-axis inertial measurement unit (IMU), a microcontroller unit, a wireless transceiver module, and a power management module. The nine-axis IMU synchronously acquires three-axis acceleration, three-axis angular velocity, and three-axis magnetic field strength. The microcontroller unit performs sampling control, zero-bias correction, temperature compensation, data encapsulation, node status detection, and low-power management. The wireless transceiver module enables bidirectional communication with the data transceiver base station in the 2.4 GHz ISM band. The power management module manages lithium battery charging and discharging, regulates power supply voltage, detects battery level, and provides anomaly protection. Each sensing node has a unique node number and includes a local timestamp, a global frame number, and battery status information in its uplink data frames.

[0040] The wireless inertial sensing nodes adopt a standardized human body layout combining front and back views. The front view includes the head, chest, left upper arm, left forearm, left hand, right upper arm, right forearm, right hand, pelvis, left thigh, left calf, right thigh, and right calf; the back view includes the back, waist, back of the left thigh, and back of the left calf. Each node is fixed to the corresponding part of the human body via a mounting bracket, elastic strap, or clothing insert, and is calibrated according to a unified node coordinate system during installation.

[0041] Step S102: Perform multi-node clock synchronization correction on the human motion data to obtain time-synchronized multi-node motion data.

[0042] In this step, when each wireless inertial sensor node collects human motion data, the local timestamp and global frame number corresponding to the collection time are appended to the data frame corresponding to each sensor node, and the data frame is sent to the data transceiver base station.

[0043] The data transceiver base station broadcasts synchronization messages to each wireless inertial sensor node at a preset period. Each wireless inertial sensor node records the reception time of the synchronization message and transmits the time information back to the data transceiver base station.

[0044] The data transceiver base station estimates the clock deviation and drift rate of each wireless inertial sensor node based on the transmission time of the synchronization message, the reception time of the synchronization message recorded by each wireless inertial sensor node, and the round-trip delay in the return time information of each wireless inertial sensor node.

[0045] The data transceiver base station corrects the local timestamps in the data frames reported by each wireless inertial sensor node based on the clock deviation and drift rate of each wireless inertial sensor node, so that the timestamps of each wireless inertial sensor node are unified with the clock reference of the data transceiver base station, thus obtaining multi-node motion data after time synchronization.

[0046] Step S103: Perform attitude calculation on the motion data of each sensor node in the time-synchronized multi-node motion data to generate attitude estimates for each sensor node. Using the current accelerometer data and magnetometer data as observations, recursively correct the attitude estimates using extended Kalman filtering, and output the extended Kalman filter attitude estimates for each sensor node.

[0047] In this step, the current triaxial angular velocity data is used as input, and the time is recursively calculated using the quaternion kinematic equations to obtain the initial estimate of the attitude quaternion at the current moment.

[0048] Acquire triaxial acceleration data within a preset time window prior to the current moment, calculate the magnitude variance of the triaxial acceleration data within the preset time window, and determine whether the magnitude variance is less than a preset variance threshold. The magnitude variance is used to characterize the intensity of motion of the current sensing node within the preset time window.

[0049] When the variance of the modulus is less than the preset variance threshold, it is determined that the current sensing node is in a low acceleration motion state. The gravity direction extracted from the triaxial acceleration data is used as the observation value, and the initial estimate of the attitude quaternion is corrected for the gravity direction to obtain the attitude estimate of the current sensing node.

[0050] When the magnitude variance is not less than the preset variance threshold, the current sensing node is determined to be in a high-acceleration motion state. The deviation between the magnitude of the triaxial acceleration data and the gravitational acceleration is used as an acceleration unreliability index. A correction intensity attenuation coefficient is generated based on the acceleration unreliability index, and partial correction is performed on the initial estimate of the attitude quaternion based on the correction intensity attenuation coefficient to obtain the attitude estimate of the current sensing node, expressed as:

[0051] ,

[0052] ,

[0053] In the formula, This is the current pose estimate of the sensor node. For unitized operations, This is the attitude quaternion correction estimate. This is the initial estimate of the attitude quaternion. This is the actual rotation angle. The correction axis is the direction vector perpendicular to the plane subtended by the estimated gravity direction vector and the observed gravity direction. This refers to quaternion multiplication operations.

[0054] Furthermore, the current attitude quaternion estimate is... As the state vector of the extended Kalman filter The state vector It is a four-dimensional vector;

[0055] An accelerometer observation model and a magnetometer observation model are constructed, wherein the expression of the accelerometer observation model is:

[0056] ,

[0057] In the formula, This is the vector of three-axis accelerometer measurements. , These are the accelerometer measurements in three axial directions, respectively. The attitude quaternion estimate at the current time. The corresponding rotation matrix, It is the transpose symbol. Let be the gravitational acceleration vector in the world coordinate system. This is the observation noise vector of the accelerometer. , These are the observation noise components for the three axes of the accelerometer;

[0058] The expression for the magnetometer observation model is:

[0059] ,

[0060] In the formula, This is the vector of three-axis measurements from the magnetometer. , These are the measurements taken by the magnetometer in three axial directions. This is the local geomagnetic field reference vector, pre-calibrated based on the geomagnetic information at the location of the sensing node. This represents the observation noise vector of the magnetometer;

[0061] Quaternion estimate at the current time position As the predicted state value, the extended Kalman filter prediction step is performed, expressed as:

[0062] ,

[0063] ,

[0064] In the formula, The current state prediction value is obtained based on the state prediction of the previous time step. This is the state estimate updated at the previous time step, i.e., the attitude quaternion estimate at the previous time step. Here is the state transition matrix. To obtain the current state covariance prediction matrix in the prediction step, Let $\mathbf{v}$ be the updated state covariance matrix from the previous time step. The process noise covariance matrix;

[0065] Accelerometer measurement at the current moment and magnetometer measurement values As an observation The extended Kalman filter update step is performed, and the expression is:

[0066] ,

[0067] ,

[0068] ,

[0069] In the formula, Here is the Kalman gain matrix. For the observation matrix, To observe the noise covariance matrix, To perform the matrix inversion operation, This is the updated state estimate at the current moment, i.e., the extended Kalman filter attitude estimate at the current moment. These are actual observed values. To predict the state Calculated predictive observations Let be the updated state covariance matrix at the current time. It is a 4×4 identity matrix;

[0070] The results of the extended Kalman filter update step are used as the extended Kalman filter attitude estimates for each sensor node.

[0071] It should be noted that the value of the observation noise covariance matrix is ​​dynamically adjusted based on the current sensor data quality. The specific adjustment method is as follows:

[0072] Calculate the magnitude deviation between the accelerometer reading and the acceleration due to gravity at the current moment. The expression is:

[0073] ,

[0074] ,

[0075] In the formula, The modulus of the accelerometer measurement. These are the measured values ​​of the three axes of the accelerometer at the current moment. This is the gravitational acceleration vector in the world coordinate system;

[0076] According to the acceleration deviation Determine the noise covariance of accelerometer observations The expression is:

[0077] ,

[0078] In the formula, Based on the noise covariance of the accelerometer observations, The preset acceleration deviation threshold, This is the preset upper limit for the noise covariance of accelerometer observations;

[0079] Calculate the magnetic field modulus deviation between the current magnetometer measurement and the reference geomagnetic field modulus. The expression is:

[0080] ,

[0081] ,

[0082] In the formula, The modulus of the magnetometer measurement. These are the measured values ​​of the three axes of the magnetometer at the current moment. Reference geomagnetic field vector The modulus length;

[0083] According to the aforementioned magnetic field modulus deviation And the magnetic field direction deviation angle between the current magnetometer measurement direction and the reference geomagnetic field direction. Determine the magnetometer observation noise covariance The expression is:

[0084] ,

[0085] ,

[0086] In the formula, Based on the noise covariance of the fundamental magnetometer observations, The magnetometer observation noise covariance under magnetic interference conditions. For magnetic interference to enter the mode length deviation threshold, To restore the modulus deviation threshold, To restore the directional deviation threshold, The threshold for the direction deviation of magnetic interference entry. It is the inverse cosine function;

[0087] The noise covariance was observed using the accelerometer. and the magnetometer observation noise covariance The observation noise covariance matrix is ​​constructed using diagonal elements.

[0088] Step S104: Perform adaptive complementary filtering secondary correction on the extended Kalman filter attitude estimate to obtain the final attitude estimate of each sensor node.

[0089] In this step, the extended Kalman filter attitude estimates quaternions of the adjacent sensor nodes corresponding to the adjacent segments of the current sensor node are obtained. The adjacent limb segment is the limb segment that is connected to the limb segment where the current sensing node is located through a single joint;

[0090] Based on the joint degree-of-freedom constraint relationship between the limb segment where the current sensing node is located and the adjacent limb segments in the human kinematics model, the number of degrees of freedom of the joint, the rotation axis direction of each degree of freedom, and the angle limit range of each degree of freedom are determined. The joint degree-of-freedom constraint relationship includes the number of rotational degrees of freedom, the rotation axis direction of each rotational degree of freedom, and the maximum allowable rotation angle of each rotational degree of freedom.

[0091] The quaternion of the extended Kalman filter attitude estimate of the current sensing node Quaternions of the extended Kalman filter attitude estimates of adjacent sensor nodes As input, calculate the relative rotation quaternion between the adjacent limb segments. ,in, The conjugate quaternion of the extended Kalman filter attitude estimates of the adjacent sensing nodes. This refers to quaternion multiplication operations;

[0092] The relative rotation quaternion Decompose it into rotational components in each degree of freedom direction, and determine whether each rotational component exceeds the corresponding angular limit range;

[0093] When all rotational components are within their corresponding angular limits, the quaternion of the extended Kalman filter attitude estimate of the current sensing node is used. Output as the final attitude estimate of the current sensing node;

[0094] When any rotation component exceeds the corresponding angular limit range, the excess rotation component is truncated to the boundary value of the corresponding angular limit range, and the corrected relative rotation quaternion is reconstructed using the truncated rotation component. According to the corrected relative rotation quaternion Quaternions of the extended Kalman filter attitude estimates of adjacent sensor nodes Back-engineering the corrected attitude estimate of the current sensor node The corrected attitude estimate of the current sensor node This is output as the final attitude estimate of the current sensor node.

[0095] Step S105: Based on the final posture estimates of each sensor node and combined with the parent-child hierarchical relationship between each limb segment in the human kinematics model, perform forward kinematic reconstruction along the tree kinematic chain with the pelvic node as the root node to generate whole-body motion posture data.

[0096] In this step, the final attitude estimates of each sensor node are quaternions. Transform from the sensor node coordinate system to the human limb coordinate system to obtain the limb quaternions of each limb segment in the human limb coordinate system. The expression is:

[0097] ,

[0098] In the formula, The quaternion is used to correct the installation orientation of each sensor node under the initial calibration attitude. This refers to quaternion multiplication operations;

[0099] Based on the parent-child hierarchical relationship between limb segments in the human kinematic model, with the pelvic node as the root node, the relative rotation quaternion of each child limb segment relative to its parent limb segment is calculated level by level along the tree kinematic chain. The expression is:

[0100] ,

[0101] In the formula, The quaternion of the parent segment. For the sub-segment quaternion, It is the conjugate quaternion of the parent segment segment quaternion;

[0102] According to the relative rotation quaternion Calculate the joint rotation angles of each joint, including the roll angle. Pitch angle and heading angle The joint rotation angle is calculated by using the relative rotation quaternion. To convert to Euler angles, the conversion formula is:

[0103] ,

[0104] ,

[0105] ,

[0106] in, These are the scalar and vector parts of the relative rotation quaternion, respectively. It is the arctangent function in the fourth quadrant. It is the arcsine function;

[0107] The three-dimensional spatial position of the pelvic nodes in the global coordinate system The starting position is determined by preset initial stance parameters. The direction vector and length of each limb segment are accumulated step-by-step along the tree-like kinematic chain to calculate the three-dimensional spatial position of each limb end in the global coordinate system. The expression for the limb end position is:

[0108] ,

[0109] In the formula, This refers to the distal end of the father limb segment. This is the position of the distal end of the female limb segment. The quaternion of the limb The corresponding rotation matrix, The length of the limb segment The unit direction vector of the limb segment in the human limb segment coordinate system;

[0110] The joint rotation angles of each joint and the three-dimensional spatial positions of each limb segment in the global coordinate system are arranged in chronological order to generate the whole-body motion posture data.

[0111] In summary, the method of this application acquires human motion data collected by multiple wireless inertial sensor nodes; performs multi-node clock synchronization correction to obtain time-synchronized multi-node motion data; performs attitude calculation on each sensor node to generate attitude estimates, and uses accelerometer and magnetometer data as observations to recursively correct the attitude estimates using extended Kalman filtering; performs adaptive complementary filtering secondary correction on the extended Kalman filter attitude estimates, and independently adjusts the correction weights of roll / pitch and heading based on acceleration and magnetic field reliability; based on the final attitude estimates of each sensor node, performs forward kinematic reconstruction along a tree-like kinematic chain with the pelvic node as the root node to generate whole-body motion attitude data; achieves high-precision and stable estimation of heading angle and accurate multi-node clock synchronization in complex environments, improving the accuracy and robustness of attitude estimation under dynamic motion conditions.

[0112] Please see Figure 2 The diagram shows a structural block diagram of a fully wireless human motion capture system based on an inertial measurement unit according to this application.

[0113] like Figure 2As shown, the fully wireless human motion capture system 200 includes an acquisition module 210, a first correction module 220, a calculation module 230, a second correction module 240, and a generation module 250.

[0114] The system includes: an acquisition module 210 configured to acquire human motion data collected by multiple wireless inertial sensor nodes; a first correction module 220 configured to perform multi-node clock synchronization correction on the human motion data to obtain time-synchronized multi-node motion data; a calculation module 230 configured to perform attitude calculation on the motion data of each sensor node in the time-synchronized multi-node motion data to generate attitude estimates for each sensor node, and to recursively correct the attitude estimates using extended Kalman filtering with current accelerometer and magnetometer data as observations, outputting extended Kalman filter attitude estimates for each sensor node; a second correction module 240 configured to perform adaptive complementary filtering secondary correction on the extended Kalman filter attitude estimates to obtain final attitude estimates for each sensor node; and a generation module 250 configured to perform forward kinematic reconstruction along a tree-like kinematic chain with the pelvic node as the root node, based on the final attitude estimates of each sensor node and the parent-child hierarchical relationship between limbs in the human kinematic model, to generate whole-body motion attitude data.

[0115] It should be understood that Figure 2 The modules and references described in the document Figure 1 The steps described in the text correspond to those in the method described above. Therefore, the operations, features, and corresponding technical effects described above also apply to the method described in the text. Figure 2 The various modules in the document will not be described in detail here.

[0116] In other embodiments, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the program instructions are executed by a processor, the processor performs the fully wireless human motion capture method based on an inertial measurement unit in any of the above method embodiments.

[0117] In one embodiment, the computer-readable storage medium of the present invention stores computer-executable instructions, which are configured as follows:

[0118] Acquire human motion data collected by multiple wireless inertial sensor nodes;

[0119] Perform multi-node clock synchronization correction on the human motion data to obtain time-synchronized multi-node motion data;

[0120] Attitude calculation is performed on the motion data of each sensor node in the multi-node motion data after time synchronization to generate attitude estimation values ​​for each sensor node. The current accelerometer data and magnetometer data are used as observations, and the attitude estimation values ​​are recursively corrected by extended Kalman filtering to output the extended Kalman filter attitude estimation values ​​for each sensor node.

[0121] An adaptive complementary filter secondary correction is performed on the extended Kalman filter attitude estimate to obtain the final attitude estimate of each sensor node.

[0122] Based on the final posture estimates of each sensor node, and combined with the parent-child hierarchical relationship between each limb segment in the human kinematics model, positive kinematic reconstruction is performed along the tree kinematic chain with the pelvic node as the root node to generate whole-body motion posture data.

[0123] Computer-readable storage media may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application program required for at least one function; the data storage area may store data created based on the use of the inertial measurement unit-based fully wireless human motion capture system, etc. Furthermore, the computer-readable storage medium may include high-speed random access memory, and may also include memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the computer-readable storage medium may optionally include memory remotely configured relative to a processor, which can be connected to the inertial measurement unit-based fully wireless human motion capture system via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0124] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present invention, such as... Figure 3 As shown, the device includes a processor 310 and a memory 320. The electronic device may also include an input device 330 and an output device 340. The processor 310, memory 320, input device 330, and output device 340 can be connected via a bus or other means. Figure 3 Taking a bus connection as an example, the memory 320 is the computer-readable storage medium described above. The processor 310 executes various server functions and data processing by running non-volatile software programs, instructions, and modules stored in the memory 320, thereby realizing the fully wireless human motion capture method based on an inertial measurement unit (IMU) described in the above embodiment. The input device 330 can receive input digital or character information and generate key signal inputs related to user settings and function control of the fully wireless human motion capture system based on an IMU. The output device 340 may include a display screen or other display device.

[0125] The aforementioned electronic device can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in the embodiments of the present invention.

[0126] In one implementation, the above-described electronic device is applied to a fully wireless human motion capture system based on an inertial measurement unit (IMU) for use as a client, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0127] Acquire human motion data collected by multiple wireless inertial sensor nodes;

[0128] Perform multi-node clock synchronization correction on the human motion data to obtain time-synchronized multi-node motion data;

[0129] Attitude calculation is performed on the motion data of each sensor node in the multi-node motion data after time synchronization to generate attitude estimation values ​​for each sensor node. The current accelerometer data and magnetometer data are used as observations, and the attitude estimation values ​​are recursively corrected by extended Kalman filtering to output the extended Kalman filter attitude estimation values ​​for each sensor node.

[0130] An adaptive complementary filter secondary correction is performed on the extended Kalman filter attitude estimate to obtain the final attitude estimate of each sensor node.

[0131] Based on the final posture estimates of each sensor node, and combined with the parent-child hierarchical relationship between each limb segment in the human kinematics model, positive kinematic reconstruction is performed along the tree kinematic chain with the pelvic node as the root node to generate whole-body motion posture data.

[0132] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fully wireless human motion capture method based on an inertial measurement unit, characterized in that, include: Acquire human motion data collected by multiple wireless inertial sensor nodes; Perform multi-node clock synchronization correction on the human motion data to obtain time-synchronized multi-node motion data; Attitude calculation is performed on the motion data of each sensor node in the multi-node motion data after time synchronization to generate attitude estimation values ​​for each sensor node. The current accelerometer data and magnetometer data are used as observations, and the attitude estimation values ​​are recursively corrected by extended Kalman filtering to output the extended Kalman filter attitude estimation values ​​for each sensor node. An adaptive complementary filter secondary correction is performed on the extended Kalman filter attitude estimate to obtain the final attitude estimate of each sensor node. Based on the final posture estimates of each sensor node, and combined with the parent-child hierarchical relationship between each limb segment in the human kinematics model, positive kinematic reconstruction is performed along the tree kinematic chain with the pelvic node as the root node to generate whole-body motion posture data.

2. The fully wireless human motion capture method based on an inertial measurement unit according to claim 1, characterized in that, The step of performing multi-node clock synchronization correction on the human motion data to obtain time-synchronized multi-node motion data includes: When collecting human motion data at each wireless inertial sensor node, the local timestamp and global frame number corresponding to the collection time are appended to the data frame corresponding to each sensor node, and the data frame is sent to the data transceiver base station. The data transceiver base station broadcasts synchronization messages to each wireless inertial sensor node at a preset period. Each wireless inertial sensor node records the reception time of the synchronization message and transmits the time information back to the data transceiver base station. The data transceiver base station estimates the clock deviation and drift rate of each wireless inertial sensor node based on the transmission time of the synchronization message, the reception time of the synchronization message recorded by each wireless inertial sensor node, and the round-trip delay in the return time information of each wireless inertial sensor node. The data transceiver base station corrects the local timestamps in the data frames reported by each wireless inertial sensor node based on the clock deviation and drift rate of each wireless inertial sensor node, so that the timestamps of each wireless inertial sensor node are unified with the clock reference of the data transceiver base station, thus obtaining multi-node motion data after time synchronization.

3. The fully wireless human motion capture method based on an inertial measurement unit according to claim 1, characterized in that, The human motion data includes triaxial acceleration data, triaxial angular velocity data, and triaxial magnetic field strength data synchronously collected by each sensor node; The process of performing attitude calculation on the motion data of each sensor node in the time-synchronized multi-node motion data to generate attitude estimates for each sensor node includes: Using the current triaxial angular velocity data as input, the initial estimate of the attitude quaternion is obtained by time recursion through the quaternion kinematic equations. Acquire triaxial acceleration data within a preset time window prior to the current moment, calculate the magnitude variance of the triaxial acceleration data within the preset time window, and determine whether the magnitude variance is less than a preset variance threshold. The magnitude variance is used to characterize the intensity of motion of the current sensing node within the preset time window. When the variance of the modulus is less than the preset variance threshold, it is determined that the current sensing node is in a low acceleration motion state. The gravity direction extracted from the triaxial acceleration data is used as the observation value, and the initial estimate of the attitude quaternion is corrected for the gravity direction to obtain the attitude estimate of the current sensing node. When the magnitude variance is not less than the preset variance threshold, the current sensing node is determined to be in a high-acceleration motion state. The deviation between the magnitude of the triaxial acceleration data and the gravitational acceleration is used as an acceleration unreliability index. A correction intensity attenuation coefficient is generated based on the acceleration unreliability index, and partial correction is performed on the initial estimate of the attitude quaternion based on the correction intensity attenuation coefficient to obtain the attitude estimate of the current sensing node, expressed as: ， ， In the formula, This is the current pose estimate of the sensor node. For unitized operations, This is the attitude quaternion correction estimate. This is the initial estimate of the attitude quaternion. This is the actual rotation angle. The correction axis is the direction vector perpendicular to the plane subtended by the estimated gravity direction vector and the observed gravity direction. This refers to quaternion multiplication operations.

4. The fully wireless human motion capture method based on an inertial measurement unit according to claim 1, characterized in that, The process uses the current accelerometer and magnetometer data as observations, and employs an extended Kalman filter to recursively correct the attitude estimate. The output extended Kalman filter attitude estimate for each sensor node includes: The current attitude quaternion estimate As the state vector of the extended Kalman filter The state vector It is a four-dimensional vector; An accelerometer observation model and a magnetometer observation model are constructed, wherein the expression of the accelerometer observation model is: ， In the formula, This is the vector of three-axis accelerometer measurements. , These are the accelerometer measurements in three axial directions, respectively. The attitude quaternion estimate at the current time. The corresponding rotation matrix, It is the transpose symbol. Let be the gravitational acceleration vector in the world coordinate system. This is the observation noise vector of the accelerometer. , These are the observation noise components for the three axes of the accelerometer; The expression for the magnetometer observation model is: ， In the formula, This is the vector of three-axis measurements from the magnetometer. , These are the measurements taken by the magnetometer in three axial directions. This is the local geomagnetic field reference vector, pre-calibrated based on the geomagnetic information at the location of the sensing node. This represents the observation noise vector of the magnetometer; Quaternion estimate at the current time position As the predicted state value, the extended Kalman filter prediction step is performed, expressed as: ， ， In the formula, The current state prediction value is obtained based on the state prediction of the previous time step. This is the state estimate updated at the previous time step, i.e., the attitude quaternion estimate at the previous time step. Here is the state transition matrix. To obtain the current state covariance prediction matrix in the prediction step, Let $\mathbf{v}$ be the updated state covariance matrix from the previous time step. The process noise covariance matrix; Accelerometer measurement at the current moment and magnetometer measurement values As an observation The extended Kalman filter update step is performed, and the expression is: ， ， ， In the formula, Here is the Kalman gain matrix. For the observation matrix, To observe the noise covariance matrix, To perform the matrix inversion operation, This is the updated state estimate at the current moment, i.e., the extended Kalman filter attitude estimate at the current moment. These are actual observed values. To predict the state Calculated predictive observations Let be the updated state covariance matrix at the current time. It is a 4×4 identity matrix; The results of the extended Kalman filter update step are used as the extended Kalman filter attitude estimates for each sensor node.

5. A fully wireless human motion capture method based on an inertial measurement unit according to claim 4, characterized in that, The value of the observation noise covariance matrix is ​​dynamically adjusted based on the sensor data quality at the current moment. The specific adjustment method is as follows: Calculate the magnitude deviation between the accelerometer reading and the acceleration due to gravity at the current moment. The expression is: ， ， In the formula, The modulus of the accelerometer measurement. These are the measured values ​​of the three axes of the accelerometer at the current moment. This is the gravitational acceleration vector in the world coordinate system; According to the acceleration deviation Determine the noise covariance of accelerometer observations The expression is: ， In the formula, Based on the noise covariance of the accelerometer observations, The preset acceleration deviation threshold, This is the preset upper limit for the noise covariance of accelerometer observations; Calculate the magnetic field modulus deviation between the current magnetometer measurement and the reference geomagnetic field modulus. The expression is: ， ， In the formula, The modulus of the magnetometer measurement. These are the measured values ​​of the three axes of the magnetometer at the current moment. Reference geomagnetic field vector The modulus length; According to the aforementioned magnetic field modulus deviation And the magnetic field direction deviation angle between the current magnetometer measurement direction and the reference geomagnetic field direction. Determine the magnetometer observation noise covariance The expression is: ， ， In the formula, Based on the noise covariance of the fundamental magnetometer observations, The magnetometer observation noise covariance under magnetic interference conditions. For magnetic interference to enter the mode length deviation threshold, To restore the modulus deviation threshold, To restore the directional deviation threshold, The threshold for the direction deviation of magnetic interference entry. It is the inverse cosine function; The noise covariance was observed using the accelerometer. and the magnetometer observation noise covariance The observation noise covariance matrix is ​​constructed using diagonal elements.

6. A fully wireless human motion capture method based on an inertial measurement unit according to claim 5, characterized in that, The process of performing adaptive complementary filtering secondary correction on the extended Kalman filter attitude estimates to obtain the final attitude estimates for each sensor node includes: Obtain the extended Kalman filter attitude estimate quaternion of the adjacent sensor nodes corresponding to the adjacent limb segments of the current sensor node. The adjacent limb segment is the limb segment that is connected to the limb segment where the current sensing node is located through a single joint; Based on the joint degree-of-freedom constraint relationship between the limb segment where the current sensing node is located and the adjacent limb segments in the human kinematics model, the number of degrees of freedom of the joint, the rotation axis direction of each degree of freedom, and the angle limit range of each degree of freedom are determined. The joint degree-of-freedom constraint relationship includes the number of rotational degrees of freedom, the rotation axis direction of each rotational degree of freedom, and the maximum allowable rotation angle of each rotational degree of freedom. The quaternion of the extended Kalman filter attitude estimate of the current sensing node Quaternions of the extended Kalman filter attitude estimates of adjacent sensor nodes As input, calculate the relative rotation quaternion between the adjacent limb segments. ,in, The conjugate quaternion of the extended Kalman filter attitude estimates of the adjacent sensing nodes. This refers to quaternion multiplication operations; The relative rotation quaternion Decompose it into rotational components in each degree of freedom direction, and determine whether each rotational component exceeds the corresponding angular limit range; When all rotational components are within their corresponding angular limits, the quaternion of the extended Kalman filter attitude estimate of the current sensing node is used. Output as the final attitude estimate of the current sensing node; When any rotation component exceeds the corresponding angular limit range, the excess rotation component is truncated to the boundary value of the corresponding angular limit range, and the corrected relative rotation quaternion is reconstructed using the truncated rotation component. According to the corrected relative rotation quaternion Quaternions of the extended Kalman filter attitude estimates of adjacent sensor nodes Back-engineering the corrected attitude estimate of the current sensor node The corrected attitude estimate of the current sensor node This is output as the final attitude estimate of the current sensor node.

7. The fully wireless human motion capture method based on an inertial measurement unit according to claim 1, characterized in that, The process involves generating full-body motion posture data by performing forward kinematic reconstruction along a tree-like kinematic chain, based on the final posture estimates of each sensor node and the parent-child hierarchical relationship between limb segments in the human kinematic model, with the pelvic node as the root node. This process includes: The final attitude estimate of each sensor node is quaternion. Transform from the sensor node coordinate system to the human limb coordinate system to obtain the limb quaternions of each limb segment in the human limb coordinate system. The expression is: ， In the formula, The quaternion is used to correct the installation orientation of each sensor node under the initial calibration attitude. This refers to quaternion multiplication operations; Based on the parent-child hierarchical relationship between limb segments in the human kinematic model, with the pelvic node as the root node, the relative rotation quaternion of each child limb segment relative to its parent limb segment is calculated level by level along the tree kinematic chain. The expression is: ， In the formula, The quaternion of the parent segment. For the sub-segment quaternion, It is the conjugate quaternion of the parent segment segment quaternion; According to the relative rotation quaternion Calculate the joint rotation angles of each joint, including the roll angle. Pitch angle and heading angle The joint rotation angle is calculated by using the relative rotation quaternion. To convert to Euler angles, the conversion formula is: ， ， ， in, These are the scalar and vector parts of the relative rotation quaternion, respectively. It is the arctangent function in the fourth quadrant. It is the arcsine function; The three-dimensional spatial position of the pelvic nodes in the global coordinate system The starting position is determined by preset initial stance parameters. The direction vector and length of each limb segment are accumulated step-by-step along the tree-like kinematic chain to calculate the three-dimensional spatial position of each limb end in the global coordinate system. The expression for the limb end position is: ， In the formula, This refers to the distal end of the father limb segment. This is the position of the distal end of the female limb segment. The quaternion of the limb The corresponding rotation matrix, The length of the limb segment The unit direction vector of the limb segment in the human limb segment coordinate system; The joint rotation angles of each joint and the three-dimensional spatial positions of each limb segment in the global coordinate system are arranged in chronological order to generate the whole-body motion posture data.

8. A fully wireless human motion capture system based on an inertial measurement unit, characterized in that, include: The acquisition module is configured to acquire human motion data collected by multiple wireless inertial sensor nodes; The first correction module is configured to perform multi-node clock synchronization correction on the human motion data to obtain time-synchronized multi-node motion data. The calculation module is configured to perform attitude calculation on the motion data of each sensor node in the multi-node motion data after time synchronization, generate attitude estimation values ​​for each sensor node, and use the accelerometer data and magnetometer data at the current moment as observations, and recursively correct the attitude estimation values ​​using extended Kalman filtering, and output extended Kalman filter attitude estimation values ​​for each sensor node. The second correction module is configured to perform adaptive complementary filtering secondary correction on the extended Kalman filter attitude estimate to obtain the final attitude estimate of each sensing node. The generation module is configured to perform forward kinematic reconstruction along the tree kinematic chain with the pelvic node as the root node, based on the final posture estimates of each sensor node and the parent-child hierarchical relationship between each limb segment in the human kinematic model, to generate whole-body motion posture data.

9. An electronic device, characterized in that, include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method according to any one of claims 1 to 7.

Images