A calibration method for inertial sensors and limbs in a human body pose capture system

Abstract

The present invention discloses a method for calibrating inertial sensors and limbs in a human posture capture system, comprising the following steps: Step 1) establishing a human anatomical model and the required coordinate system, calculating the conversion quaternion between a binocular camera and a geographic coordinate system, and performing distortion calibration and binocular calibration; Step 2) placing N inertial sensors on the human body, wearing them, and standing or sitting in front of a binocular camera, with each left and right camera simultaneously capturing an image; Step 3) using the two images to calculate the three-dimensional coordinates of the joint nodes, and calculating the limb posture using these coordinates and joint constraints; Step 4) calculating the posture of the limb in the geographic coordinate system using the conversion quaternion between the camera and the geographic coordinate system. This method no longer relies on traditional fixed calibration movements, but instead uses vision to directly measure the human body's posture, reducing the accuracy requirements of the movement in the calibration method.

Description

Technical Field

[0001] The present invention belongs to the technical field of inertial motion capture, in particular to the calibration of inertial sensors and human limbs in inertial motion capture. Background Art

[0002] In inertial motion capture, inertial sensors are typically attached to human limbs, with one or two sensors per limb. These sensors move with the body and output their own posture data. Due to factors such as inaccurate placement of the inertial sensors and the presence of muscle between the bones and skin, there can be a deviation between the posture of the inertial sensors and the limb. This deviation is somewhat random, making the data directly output by the inertial sensors inaccurate in reflecting the body's motion. Therefore, a calibration method is needed to establish a relationship between the posture output by the inertial sensors and the body's motion, ensuring that each sensor's posture accurately reflects the limb's motion. Calibration methods involve two uncertainties: uncertainty in the body's posture and uncertainty in the relationship between the inertial sensors and the limb. In typical calibration methods, several specific motions are selected, whether dynamic or static, to fix the body's posture at a specific moment in time as a known posture. This allows the inherently uncertain body posture to be determined. This allows the relationship between the inertial sensors and the limb to be calculated using the inertial sensor and body posture data. This method proposes another idea. Instead of determining the human body posture through fixed movements, the human body posture can be directly obtained through visual methods. It is more convenient and quick, and is friendly to people with certain special diseases. Summary of the Invention

[0003] In order to simplify the difficulty of calibrating the inertial sensor posture and human limb posture in inertial motion capture, the present invention proposes a method for calibrating inertial sensors and limbs in a human posture capture system, which greatly simplifies the calibration steps and can meet the needs of people with special diseases.

[0004] To achieve the above-mentioned object of the invention, the present invention provides a method for calibrating an inertial sensor and a limb in a human posture capture system, comprising the following steps:

[0005] Step 1) Establish the human anatomy model and the required coordinate system, which includes the geographic coordinate system, the inertial sensor coordinate system and the camera coordinate system. Calculate the conversion quaternion q between the binocular camera and the geographic coordinate system CF-GF , the camera is placed at a certain angle or parallel, and distortion calibration and binocular calibration are performed.

[0006] Step 2) Place N inertial sensors on the human body as needed, one on each limb. The inertial sensors are not required in any specific pose. After wearing them, the user stands or sits in front of a binocular camera. The left and right cameras simultaneously capture an image, each including all limbs equipped with the inertial sensors.

[0007] Step 3) The two images are each estimated using a human pose estimation algorithm to estimate the position of each joint node. Based on triangulation, the three-dimensional coordinates of the joint nodes can be calculated using the two images, and the limb pose can be calculated using the coordinates and joint constraints. CF q SG-C .

[0008] Step 4) Convert the quaternion q between the camera and the geographic coordinate system CF-GF Calculate the posture of the limbs in the geographic coordinate system This attitude is consistent with the attitude of the inertial sensor in the geographic coordinate system GF q SG-I There is a deviation, which is expressed as a quaternion The correct posture of the limb after calibration is obtained by multiplying the quaternion with the quaternion output by the inertial sensor.

[0009] This invention estimates human posture through binocular vision and uses binocular triangulation to calculate the three-dimensional coordinates of human joints. The desired limb posture is calculated using the three-dimensional coordinates of the joints and joint constraints. This posture deviation is then directly calculated using the limb posture and the posture output from the inertial sensor. This method no longer relies on traditional fixed calibration motions, but instead uses vision to directly measure human posture, reducing the accuracy requirements of the calibration method. The subject can choose any posture for calibration, including patients with certain special conditions, expanding the range of patients suitable for inertial motion capture. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flow chart of the present invention;

[0011] Figure 2 is the human anatomy model used in this embodiment;

[0012] Figure 3 Schematic diagram of the arrangement of the inertial motion capture system and binocular camera in this embodiment;

[0013] Figure 4 is a schematic diagram of the estimation of limb posture in the camera coordinate system in this embodiment;

[0014] Figure 5 Schematic diagram of solving calibration deviation in this embodiment. DETAILED DESCRIPTION

[0015] To make the technical solution and design idea of the present invention clearer, the lower body of the human body is selected as the posture estimation object, and two visual cameras and six inertial sensors are used. The present invention is further described in conjunction with the accompanying drawings.

[0016] refer to Figure 1 、 Figure 2 、 Figure 3 、 Figure 4 、 Figure 5 , a method for calibrating inertial sensors and limbs in a human posture capture system, the steps of which are as follows:

[0017] Step 1) Determine the required coordinate system, which includes the geographic coordinate system (G system), the inertial sensor coordinate system (I system) and the camera coordinate system (C system). Calculate the conversion quaternion q between the binocular camera and the geographic coordinate system CF-GF , the camera is placed at a certain angle or parallel, and distortion calibration and binocular calibration are performed and a human anatomy model is established. This step is specifically implemented as follows:

[0018] The geographic coordinate system is the Northeast Celestial (EUN) coordinate system. When the inertial sensor's X-axis coincides with the east, the Y-axis coincides with the north, and the Z-axis coincides with the celestial, the inertial sensor's initial attitude. Its Euler angles are all zero, and the quaternion is [0, 0, 0, 1]. Quaternions, Euler angles, and rotation matrices are all representations of attitude and are essentially the same. To avoid gimbal lock and reduce computational complexity, this paper uses quaternions to represent attitude.

[0019] When two postures are in the same reference system or the same posture is in two reference systems, there is a deviation posture, that is, a deviation quaternion. There is a conversion quaternion q from CF to GF between the camera coordinate system and the geographic coordinate system. CF-GF This conversion quaternion can be calculated using inertial sensors placed on a camera or a marker. The cameras need to be positioned at a certain angle or parallel to each other, and distortion can be eliminated through camera calibration. The positional relationship between the two cameras can then be determined through binocular calibration.

[0020] The lower limb anatomical model used in this embodiment is as follows: Figure 2 As shown, each lower limb in this anatomical model has a total of 7 degrees of freedom, and the joints are abstracted as joint points. The hip joint is a ball joint with 3 degrees of freedom, the knee joint is a hinge joint with only 1 degree of freedom, and the ankle joint is a ball joint with 3 degrees of freedom. Because the knee joint is a hinge joint, the hip, knee, and ankle joints are always in the same plane. The normal vector to this plane is the X-axis of the thigh and shank coordinate systems. The Z-axis of the thigh coordinate system is oriented from the knee to the hip joint, while the Z-axis of the shank coordinate system is oriented from the ankle to the knee joint. The Y-axis is the vector product of the Z-axis and the X-axis. The Y-axis of the foot coordinate system is oriented from the heel to the toe. The X-axis is the normal vector to the plane formed by the ankle, heel, and toe, and the Z-axis is the vector product of the X-axis and the Y-axis.

[0021] Step 2) Place N inertial sensors on the human body as needed, one on each limb. The inertial sensors are in any position. After wearing them, the user stands or sits in front of the binocular camera. The left and right cameras simultaneously capture an image, each including all limbs equipped with the inertial sensors. This step is implemented as follows:

[0022] The schematic diagram of the binocular camera and sensor used in this embodiment is as follows: Figure 3 As shown in the figure, three inertial sensors are placed on each leg—one on the thigh, one on the calf, and one on the foot. These sensors can output attitude quaternions. These three sensors are connected in series via the CAN bus, converted to a serial port via a repeater, and transmitted to the host computer in real time.

[0023] Each binocular camera takes an image, and the subject's lower limbs are required to be within the camera's field of view and maintain the same posture for 1-2 seconds, waiting for the two cameras to complete the shooting. At the same time, the inertial sensor inputs the posture data measured by it into the host computer.

[0024] Step 3) The two images are each estimated using a human pose estimation algorithm to estimate the position of each joint node. Based on triangulation, the three-dimensional coordinates of the joint nodes can be calculated using the two images, and the limb pose can be calculated using the coordinates and joint constraints. CF q SG-C The specific implementation of this step is as follows:

[0025] The output image of the binocular camera is calibrated and input into the human posture estimation algorithm to obtain the two-dimensional coordinates of the human joint nodes. The corresponding coordinates in the two images are theoretically the same coordinates. According to the triangulation principle of the binocular camera, the depth information of the required joint points can be calculated based on the known position relationship of the binocular cameras, and finally the three-dimensional information of the joint points can be obtained. This information can be combined with the human anatomical model in step 1) to calculate the posture of the lower limbs. The process is as follows Figure 4 shown.

[0026] Step 4) Convert the quaternion q between the camera and the geographic coordinate system CF-GF Calculate the posture of the limbs in the geographic coordinate system This attitude is consistent with the attitude of the inertial sensor in the geographic coordinate system GF q SG-I There is a deviation, which is expressed as a quaternion The correct posture of the limb after calibration is obtained by multiplying the quaternion with the quaternion output by the inertial sensor. The specific implementation of this step is as follows:

[0027] The limb posture calculated in step 3) needs to be transformed into quaternion in the camera coordinate system:

[0028]

[0029] Convert to geographic coordinate system, where GF q SG-C It is the posture of the lower limbs in the geographic coordinate system, and SG represents a certain limb, namely the thigh, calf and foot.

[0030] The inertial sensors on each limb can measure a posture quaternion, namely GF q SG-I , the posture and the transformed limb posture are both in the geographic coordinate system, so there is a transformation quaternion between the two postures, that is, the deviation quaternion, through:

[0031]

[0032] The calibration quaternion of each limb can be calculated CF q SG-A .

[0033] The embodiments of this specification are merely examples of implementations of the invention and are provided for illustrative purposes only. The scope of protection of the present invention should not be considered limited to the specific embodiments described in these embodiments. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by a person of ordinary skill in the art based on the invention.

Claims

1. A method for calibrating inertial sensors and limbs in a human posture capture system, characterized in that: The following steps are involved: Step 1) Establish the human anatomical model and the required coordinate system, which includes the geographic coordinate system, the inertial sensor coordinate system and the camera coordinate system; calculate the conversion quaternion q between the binocular camera and the geographic coordinate system CF-GF , the camera is placed at an angle or parallel, and distortion calibration and binocular calibration are performed; Step 2) Place N inertial sensors on the human body as needed, one on each limb. The inertial sensors are in any position. After wearing them, the user stands or sits in front of a binocular camera. The left and right cameras simultaneously capture an image, each including all limbs equipped with the inertial sensors. Step 3) The two images are each estimated using a human pose estimation algorithm to estimate the position of each joint node. Based on triangulation, the three-dimensional coordinates of the joint nodes are calculated using the two images. The limb pose is calculated using the coordinates and joint constraints. CF q SG-C ; Step 4) Convert the quaternion q between the camera and the geographic coordinate system CF-GF Calculate the posture of the limbs in the geographic coordinate system This attitude is consistent with the attitude of the inertial sensor in the geographic coordinate system GF q SG-I There is a deviation, which is expressed as a quaternion The correct posture of the limb after calibration is obtained by multiplying the quaternion with the quaternion output by the inertial sensor.

Images