Human upper limb posture estimation method and system based on inertial position tracker and depth body sensing camera

Abstract

The present invention discloses a method and system for estimating the posture of human upper limbs based on an inertial position tracker and a deep somatosensory camera. The method comprises: collecting posture signals of bone points of human upper limb skeleton by a deep somatosensory camera, and collecting posture signals of human elbow joint by an inertial position tracker; constructing a global coordinate system, and converting the posture signals of the bone points of the upper limb skeleton and the elbow joint posture signals into the global coordinate system; obtaining the positions of the elbow joint and wrist joint in the global coordinate system according to a dual Kalman filtering method, and calculating the elbow joint angle in combination with the shoulder joint position obtained according to the posture signals of the bone points of the upper limb skeleton. The present invention adopts a dual Kalman filtering algorithm to fuse the data of the inertial position tracker and the deep somatosensory camera, and makes full use of the various signal resources of the sensor, so as to meet the requirements of strong robustness and high estimation accuracy in a dynamic environment, and overcome the occlusion problem of the deep somatosensory camera.

Description

Technical Field

[0001] The present invention relates to the field of human body posture estimation, and in particular to a method and system for estimating the posture of human upper limbs, and in particular to a method and system for estimating the posture of human upper limbs based on an inertial position tracker and a depth somatosensory camera. Background Art

[0002] With the development of virtual reality, augmented reality, robot control, motion analysis and other fields, human posture estimation has become one of the key technologies in many applications. Traditional posture estimation methods mainly rely on a single sensor, such as an inertial position tracker or a visual sensor. However, a single sensor has certain limitations. For example, although the inertial position tracker can provide high-frequency real-time data, which is suitable for capturing dynamic motion, the data of the inertial position tracker is easily affected by cumulative errors, which may lead to large position estimation errors after long-term use.

[0003] Visual sensors, especially depth-sensing cameras, can provide accurate spatial position information and effectively capture the movement of limbs. However, visual sensors are limited by ambient lighting, occlusion, and depth perception range, and are easily disturbed, especially in dynamic environments. When joint positions move significantly, depth image distortion or tracking failure may occur. Especially in the pose estimation of small joints of the upper limbs, such as the wrist joint, the position estimation effect is even worse.

[0004] At present, the effective fusion of inertial sensor and depth body camera data for upper limb posture estimation can combine their respective advantages to improve the overall posture estimation accuracy and robustness. However, during fast or intense movements, the advantages of multi-source signals are not fully utilized, and there is still room for improvement in the accuracy of small joint position estimation and occlusion issues. Summary of the invention

[0005] In order to achieve high robustness of human upper limb posture estimation and overcome the needs of small joint position estimation and occlusion problems, the present invention proposes a human upper limb posture estimation method and system based on an inertial position tracker and a depth somatosensory camera.

[0006] The technical solution adopted by the present invention is as follows:

[0007] A method for estimating the posture of human upper limbs based on an inertial position tracker and a depth somatosensory camera comprises the following steps:

[0008] The depth sensing camera is used to collect the position and posture signals of the skeleton points of the human upper limbs, and the inertial position tracker is used to collect the position and posture signals of the human elbow joints;

[0009] Constructing a global coordinate system, and converting the upper limb skeleton bone point pose signal and the elbow joint pose signal into the global coordinate system;

[0010] The positions of the elbow joint and wrist joint in the global coordinate system are obtained according to the double Kalman filtering method, and the elbow joint angle is calculated in combination with the shoulder joint position obtained according to the upper limb skeleton bone point posture signal.

[0011] Furthermore, the depth somatosensory camera is placed in front of the human body, and the upper limb skeleton bone point posture signal includes shoulder joint, elbow joint and wrist joint related data; the inertial position tracker is set at the elbow joint position of the human body, and the elbow joint posture signal includes elbow joint related data.

[0012] Furthermore, the global coordinate system is constructed to convert the upper limb skeleton bone point pose signal and the elbow joint pose signal into the global coordinate system, specifically:

[0013] A global coordinate system is constructed according to the human elbow joint posture signal collected by the inertial position tracker, and a depth somatosensory camera coordinate system is constructed according to the human upper limb skeleton bone point posture signal collected by the depth somatosensory camera. The global coordinate system is parallel to the depth somatosensory camera coordinate system; according to the overlapped translation amount of the two coordinate systems, the upper limb skeleton bone point posture signal is converted to the global coordinate system.

[0014] Further, the obtaining of the elbow joint and wrist joint positions in the global coordinate system according to the dual Kalman filtering method includes:

[0015] The elbow joint position is calculated using the Kalman filter method, specifically: establishing an elbow joint state equation according to the elbow joint posture signal, establishing an elbow joint observation equation according to the upper limb skeleton bone point posture signal, and introducing a flag indicating whether the elbow joint bone point is blocked in the upper limb skeleton bone point posture signal. , predict and update the elbow joint position;

[0016] The wrist joint position is calculated using the Kalman filter method, specifically: the direction vector from the elbow to the wrist joint is established according to the predicted updated elbow joint position and the elbow joint posture signal, the direction vector state equation from the elbow to the wrist joint is established according to the elbow joint posture signal, the direction vector observation equation from the elbow to the wrist joint is established according to the upper limb skeleton bone point posture signal, and the wrist joint bone point is introduced in the upper limb skeleton bone point posture signal Whether the wrist joint bone point is blocked , predict and update the direction vector from the elbow joint to the wrist joint, and determine the wrist joint position according to the predicted and updated direction vector from the elbow joint to the wrist joint.

[0017] Furthermore, the elbow joint state equation is:

[0018] ,

[0019] Among them, the image signal frequency of the depth sensing camera is used as the time reference. is the prediction of the elbow joint position velocity signal at the kth time at the k-1th time, is the prediction of the elbow joint position signal at k-1 time to k time, is the prediction of elbow joint velocity signal from k-1 time to k time; , , is the time interval of the depth sensor camera image; is the elbow joint position velocity signal collected by the inertial position tracker at time k-1, is the elbow joint position signal collected by the k-1 time inertial position tracker, is the elbow joint velocity signal collected by the k-1 time inertial position tracker; The elbow joint acceleration signal collected by the inertial position tracker; is the system noise with zero mean;

[0020] The elbow joint observation equation is:

[0021] ,

[0022] in, , is the elbow joint position signal collected by the k-time depth somatosensory camera; is the elbow joint velocity signal collected by the k-time depth somatosensory camera; ; is the measurement noise with zero mean, and the system noise and measurement noise The corresponding covariance is incoherent Gaussian white noise.

[0023] Furthermore, the state equations of the direction vectors of the elbow joint and the wrist joint are:

[0024] ,

[0025] in, is the direction vector from the elbow joint to the wrist joint predicted at time k according to the updated elbow joint position and the elbow joint posture signal at time k-1; is the transformation matrix of the inertial position tracker from time k-1 to time k based on the depth camera time reference; A direction vector from the elbow joint to the wrist joint at time k-1 established according to the updated elbow joint position and the elbow joint posture signal at time k-1; is the system noise with zero mean;

[0026] The state equations of the elbow joint and wrist joint direction vectors are:

[0027] ,

[0028] in, The direction vectors of the elbow and wrist joints observed by the depth sensing camera; , is the identity matrix; is the measurement noise with zero mean, and the system noise and measurement noise The corresponding covariance is incoherent Gaussian white noise.

[0029] Furthermore, when the elbow joint bone point in the upper limb skeleton bone point pose signal is not blocked, , the prediction covariance matrix , according to the covariance matrix Kalman gain , elbow joint position velocity vector , covariance matrix Make updates;

[0030] When the elbow joint bone point in the upper limb skeleton bone point pose signal is blocked, , the elbow joint position velocity vector predicted by the inertial position tracker at time k-1 Updated as the elbow joint position velocity vector for the next frame.

[0031] Furthermore, when the wrist joint bone point in the upper limb skeleton bone point pose signal is not blocked, , the prediction covariance matrix , according to the covariance matrix Kalman gain , elbow and wrist joint direction vectors , covariance matrix Make updates;

[0032] When the wrist joint bone point in the upper limb skeleton bone point pose signal is blocked, , to predict the direction vector from the elbow to the wrist at time k according to the predicted updated elbow position and the elbow pose signal at time k-1 Updated as the direction vector of the elbow and wrist joints in the next frame.

[0033] Furthermore, the elbow joint angle is calculated as follows:

[0034] ,

[0035] in, is the elbow joint angle, is the joint vector from the shoulder joint to the elbow joint, The joint vector from the elbow to the wrist.

[0036] A human upper limb posture estimation system based on an inertial position tracker and a depth somatosensory camera, comprising:

[0037] Data acquisition module: used to collect the posture signals of the skeleton points of the human upper limbs through the depth sensing camera, and collect the posture signals of the human elbow joint through the inertial position tracker;

[0038] Coordinate system construction module: used to construct a global coordinate system, and convert the upper limb skeleton bone point posture signal and the elbow joint posture signal into the global coordinate system;

[0039] Data processing module: used to obtain the positions of the elbow joint and wrist joint in the global coordinate system according to the double Kalman filtering method, and calculate the elbow joint angle in combination with the shoulder joint position obtained according to the upper limb skeleton bone point posture signal.

[0040] The beneficial effects of the present invention are:

[0041] 1) The present invention utilizes an inertial position tracker and a depth sensing camera to collect posture signals, and adopts a dual Kalman filter algorithm to fuse the data of the two, thereby achieving high estimation accuracy.

[0042] 2) The present invention makes full use of various resource signals of sensors and has strong robustness in dynamic environments.

[0043] 3) The present invention makes up for the deficiency of the depth somatosensory camera in estimating the wrist joint position. When the depth somatosensory camera signal is blocked, the tracking data of the inertial position tracker is used to compensate, thus overcoming the occlusion problem. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of a specific process of the dual Kalman filtering method in an embodiment of the present invention.

[0045] Figure 2 Schematic diagram of human upper limb posture estimation in an embodiment of the present invention.

[0046] Figure 3 Schematic diagram of the inertial position tracker coordinate system, the depth sensing camera coordinate system and the global coordinate system in an embodiment of the present invention.

[0047] Figure 4 FIG. 1 is a diagram showing the wearing position of an inertial position tracker on a human upper limb in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of the present disclosure. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.

[0049] The depth sensing camera and inertial position tracker used in the present invention are both commercially available products.

[0050] A method for estimating the posture of human upper limbs based on an inertial position tracker and a depth somatosensory camera comprises the following steps:

[0051] The depth somatosensory camera is used to collect the position signals of the bone points of the human upper limb skeleton, and the inertial position tracker is used to collect the position signals of the human elbow joint. This embodiment uses the inertial position tracker and Microsoft's depth somatosensory camera as sensors, places the depth somatosensory camera in front of the human body, obtains the position signals of the bone points of the human upper limb skeleton, and wears the inertial position tracker at the human elbow joint position (such as Figure 4 As shown in the figure, the elbow joint posture signal is collected during the movement of the forearm. In this embodiment, A, B, and C are used to represent the shoulder joint, elbow joint, and wrist joint, respectively, AB represents the upper arm, BC represents the forearm, Represents the elbow joint angle.

[0052] like Figure 3 As shown, a global coordinate system is constructed to transform the upper limb skeleton bone point pose signal and the elbow joint pose signal into the global coordinate system. Specifically:

[0053] Constructing a global coordinate system based on the human elbow joint posture signal collected by the inertial position tracker , The base coordinate system of the inertial position tracker is used, and the posture of the human elbow joint is obtained according to the rotation transformation angle of the base coordinate system in the global coordinate system; the depth somatosensory camera coordinate system is constructed according to the posture signal of the human upper limb skeleton bone point collected by the depth somatosensory camera , the global coordinate system And the depth sensor camera coordinate system Maintain parallel relationship; according to the translation amount of the coincidence of the two coordinate systems , convert the upper limb skeleton point pose signal to the global coordinate system, and the conversion equation is:

[0054] ,

[0055] ,

[0056] ,

[0057] in, is the x-coordinate value in the global coordinate system, is the y-coordinate value in the global coordinate system, is the coordinate value in the z direction in the global coordinate system; is the x-coordinate value in the depth camera coordinate system, is the y-coordinate value in the depth camera coordinate system, is the coordinate value in the z direction of the depth sensing camera coordinate system; is the x-direction offset between the global coordinate system and the depth camera coordinate system, is the y-direction offset between the global coordinate system and the depth camera coordinate system, It is the z-direction offset between the global coordinate system and the depth sensing camera coordinate system.

[0058] The positions of the elbow and wrist joints in the global coordinate system are obtained by using the dual Kalman filtering method, and the elbow joint angle is calculated by combining the shoulder joint position obtained according to the upper limb skeleton bone point posture signal. The specific steps are as follows:

[0059] like Figure 1 As shown, the Kalman filter method is used to calculate the elbow joint position, specifically: establish an elbow joint state equation according to the elbow joint posture signal, establish an elbow joint observation equation according to the upper limb skeleton bone point posture signal, and introduce a flag of whether the elbow joint bone point is blocked in the upper limb skeleton bone point posture signal , predict and update the elbow joint position.

[0060] The elbow joint state equation is:

[0061] ,

[0062] Among them, the image signal frequency of the depth sensing camera is used as the time reference. is the prediction of the elbow joint position velocity signal at the kth time at the k-1th time, is the prediction of the elbow joint position signal at k-1 time to k time, is the prediction of elbow joint velocity signal from k-1 time to k time; , , is the time interval of the depth sensor camera image; is the elbow joint position velocity signal collected by the inertial position tracker at time k-1, is the elbow joint position signal collected by the k-1 time inertial position tracker, is the elbow joint velocity signal collected by the k-1 time inertial position tracker; The elbow joint acceleration signal collected by the inertial position tracker; is the system noise with zero mean;

[0063] The elbow joint observation equation is:

[0064] ,

[0065] in, , is the elbow joint position signal collected by the k-time depth somatosensory camera; is the elbow joint velocity signal collected by the k-time depth somatosensory camera; ; is the measurement noise with zero mean, and the system noise and measurement noise The corresponding covariance is incoherent Gaussian white noise.

[0066] When the elbow joint bone point in the upper limb skeleton bone point pose signal is not blocked, , the prediction covariance matrix , according to the covariance matrix Kalman gain , elbow joint position velocity vector , covariance matrix The specific prediction and update methods are as follows:

[0067] The predicted covariance matrix: ;

[0068] Kalman gain: ;

[0069] Updated elbow joint position velocity vector: ;

[0070] Updated covariance matrix: .

[0071] in, is the k-time covariance matrix predicted at time k-1, is the covariance matrix updated at time k-1, for The transpose of is the k-1 time process noise covariance matrix, is the Kalman gain at time k, is the measurement noise covariance matrix of the elbow joint position velocity signal at time k-1, for The transpose of is the elbow joint position velocity signal updated at time k, is the identity matrix.

[0072] When the elbow joint bone point in the upper limb skeleton bone point pose signal is blocked, , the elbow joint position velocity vector predicted by the inertial position tracker at time k-1 Updated as the elbow joint position velocity vector for the next frame.

[0073] The wrist joint position is calculated using the Kalman filter method, specifically: the direction vector from the elbow to the wrist joint is established according to the predicted updated elbow joint position and the elbow joint posture signal, the direction vector state equation from the elbow to the wrist joint is established according to the elbow joint posture signal, the direction vector observation equation from the elbow to the wrist joint is established according to the upper limb skeleton bone point posture signal, and the wrist joint bone point is introduced in the upper limb skeleton bone point posture signal Whether the wrist joint bone point is blocked , predict and update the direction vector from the elbow joint to the wrist joint, and determine the wrist joint position according to the predicted and updated direction vector from the elbow joint to the wrist joint.

[0074] The state equations of the elbow joint and wrist joint direction vectors are:

[0075] ,

[0076] in, is the direction vector from the elbow joint to the wrist joint predicted at time k according to the updated elbow joint position and the elbow joint posture signal at time k-1, and the direction vector modulus is ; is the distance between the elbow and wrist bone points of the depth sensing camera at the initial time; is the transformation matrix of the inertial position tracker from the last k-1th time to the kth time in the depth camera time reference, is the quaternion of the posture conversion of the elbow joint posture signal from the k-1th frame to the kth frame, is the quaternion of the elbow joint posture signal of the kth frame, is the quaternion of the elbow joint posture signal in the k-1th frame; is the direction vector from the elbow joint to the wrist joint at time k-1 established according to the updated elbow joint position and the elbow joint posture signal at time k-1; is the system noise with zero mean;

[0077] The state equations of the elbow joint and wrist joint direction vectors are:

[0078] ,

[0079] in, is the direction vector of the elbow and wrist joints observed by the depth somatosensory camera, and the elbow joint position observed by the depth somatosensory camera and wrist position , ; , is the identity matrix; is the measurement noise with zero mean, and the system noise and measurement noise The corresponding covariance and is incoherent Gaussian white noise.

[0080] When the wrist joint bone point in the upper limb skeleton bone point pose signal is not blocked, , the prediction covariance matrix , according to the covariance matrix Kalman gain , elbow and wrist joint direction vectors , covariance matrix The specific prediction and update methods are as follows:

[0081] The predicted covariance matrix: ;

[0082] Kalman gain: ;

[0083] Updated elbow-to-wrist space vector: ;

[0084] Updated covariance matrix: .

[0085] in The k-time covariance matrix predicted at time k-1, is the covariance matrix updated at time k-1, for The transpose of is the k-1 time process noise covariance matrix, is the Kalman gain at time k, for The transpose of is the measurement noise covariance matrix of the k-1 time elbow-to-wrist space vector, is the identity matrix.

[0086] When the wrist joint bone point in the upper limb skeleton bone point pose signal is blocked, , the direction vector from the elbow to the wrist at time k is predicted based on the predicted updated elbow position at time k-1 and the elbow pose signal Updated as the direction vector of the elbow and wrist joints in the next frame.

[0087] After double Kalman filtering, the elbow joint position is obtained and wrist position , combined with the wrist joint position obtained from the upper limb skeleton bone point posture signal , calculate the elbow joint angle. The elbow joint angle is calculated as follows:

[0088] ,

[0089] in, is the elbow joint angle, is the joint vector from the shoulder joint to the elbow joint, , The joint vector from the elbow to the wrist, In order to improve the robustness of the depth somatosensory camera in estimating the wrist joint position, the present invention adopts a double Kalman filter method to calculate the elbow joint position and the wrist joint position based on the combination of the inertial position tracker and the depth somatosensory camera, and introduces the elbow joint occlusion parameter and wrist occlusion parameters , in order to reduce the impact of depth sensing camera occlusion.

[0090] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 Process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0092] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0094] The above-mentioned embodiments only express the implementation mode of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. For ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for estimating the posture of human upper limbs based on an inertial position tracker and a depth somatosensory camera, characterized in that: The following steps are involved: The depth sensing camera is used to collect the position and posture signals of the skeleton points of the human upper limbs, and the inertial position tracker is used to collect the position and posture signals of the human elbow joints; Constructing a global coordinate system, and converting the upper limb skeleton bone point pose signal and the elbow joint pose signal into the global coordinate system; Specifically, a global coordinate system is constructed according to the human elbow joint posture signal collected by the inertial position tracker, and a depth somatosensory camera coordinate system is constructed according to the human upper limb skeleton bone point posture signal collected by the depth somatosensory camera, wherein the global coordinate system is parallel to the depth somatosensory camera coordinate system; and the upper limb skeleton bone point posture signal is converted into the global coordinate system according to the translation amount of the two coordinate systems; Acquire the positions of the elbow joint and the wrist joint in the global coordinate system according to the double Kalman filtering method, and calculate the elbow joint angle in combination with the shoulder joint position obtained according to the upper limb skeleton bone point posture signal; The method of obtaining the positions of the elbow joint and the wrist joint in the global coordinate system according to the dual Kalman filtering method includes: The elbow joint position is calculated using the Kalman filter method, specifically: establishing an elbow joint state equation according to the elbow joint posture signal, establishing an elbow joint observation equation according to the upper limb skeleton bone point posture signal, and introducing a flag indicating whether the elbow joint bone point is blocked in the upper limb skeleton bone point posture signal. , predict and update the elbow joint position; The wrist joint position is calculated using the Kalman filter method, specifically: the direction vector from the elbow to the wrist joint is established according to the predicted updated elbow joint position and the elbow joint posture signal, the direction vector state equation from the elbow to the wrist joint is established according to the elbow joint posture signal, the direction vector observation equation from the elbow to the wrist joint is established according to the upper limb skeleton bone point posture signal, and the wrist joint bone point is introduced in the upper limb skeleton bone point posture signal Whether the wrist joint bone point is blocked , predict and update the direction vector from the elbow joint to the wrist joint, and determine the wrist joint position according to the predicted and updated direction vector from the elbow joint to the wrist joint.

2. According to claim 1, a method for estimating the posture of human upper limbs based on an inertial position tracker and a depth sensing camera is characterized in that: The depth somatosensory camera is placed in front of the human body, and the upper limb skeleton bone point posture signal includes shoulder joint, elbow joint and wrist joint related data; the inertial position tracker is set at the elbow joint position of the human body, and the elbow joint posture signal includes elbow joint related data.

3. The method for estimating the posture of human upper limbs based on an inertial position tracker and a depth sensing camera according to claim 1, characterized in that: The elbow joint state equation is: ， Among them, the image signal frequency of the depth sensing camera is used as the time reference. is the prediction of the elbow joint position velocity signal at the kth time at the k-1th time, is the prediction of the elbow joint position signal at k-1 time to k time, is the prediction of elbow joint velocity signal from k-1 time to k time; , , is the time interval of the depth sensor camera image; is the elbow joint position velocity signal collected by the inertial position tracker at time k-1, is the elbow joint position signal collected by the k-1 time inertial position tracker, is the elbow joint velocity signal collected by the k-1 time inertial position tracker; The elbow joint acceleration signal collected by the inertial position tracker; is the system noise with zero mean; The elbow joint observation equation is: ， in, , is the elbow joint position signal collected by the k-time depth somatosensory camera; is the elbow joint velocity signal collected by the k-time depth somatosensory camera; ; is the measurement noise with zero mean, and the system noise and measurement noise The corresponding covariance is incoherent Gaussian white noise.

4. The method for estimating the posture of human upper limbs based on an inertial position tracker and a depth sensing camera according to claim 1, characterized in that: The state equations of the elbow joint and wrist joint direction vectors are: ， in, is the direction vector from the elbow joint to the wrist joint predicted at time k according to the updated elbow joint position and the elbow joint posture signal at time k-1; is the transformation matrix of the inertial position tracker from time k-1 to time k based on the depth camera time reference; is the direction vector from the elbow joint to the wrist joint at time k-1 established according to the updated elbow joint position and the elbow joint posture signal at time k-1; is the system noise with zero mean; The state equations of the elbow joint and wrist joint direction vectors are: ， in, The direction vectors of the elbow and wrist joints observed by the depth sensing camera; , is the identity matrix; is the measurement noise with zero mean, and the system noise and measurement noise The corresponding covariance is incoherent Gaussian white noise.

5. The method for estimating the posture of human upper limbs based on an inertial position tracker and a depth sensing camera according to claim 1, characterized in that: When the elbow joint bone point in the upper limb skeleton bone point pose signal is not blocked, , the prediction covariance matrix , according to the covariance matrix Kalman gain , elbow joint position velocity vector , covariance matrix Make updates; When the elbow joint bone point in the upper limb skeleton bone point pose signal is blocked, , the elbow joint position velocity vector predicted by the inertial position tracker at time k-1 Updated as the elbow joint position velocity vector for the next frame.

6. The method for estimating the posture of human upper limbs based on an inertial position tracker and a depth sensing camera according to claim 1, characterized in that: When the wrist joint bone point in the upper limb skeleton bone point pose signal is not blocked, , the prediction covariance matrix , according to the covariance matrix Kalman gain , elbow and wrist joint direction vectors , covariance matrix Make updates; When the wrist joint bone point in the upper limb skeleton bone point pose signal is blocked, , the direction vector from the elbow to the wrist at time k is predicted based on the predicted updated elbow position at time k-1 and the elbow pose signal Updated as the direction vector of the elbow and wrist joints in the next frame.

7. The method for estimating the posture of human upper limbs based on an inertial position tracker and a depth sensing camera according to claim 1, characterized in that: The calculation method of the elbow joint angle is: ， in, is the elbow joint angle, is the joint vector from the shoulder joint to the elbow joint, The joint vector from the elbow to the wrist.

8. A human upper limb posture estimation system based on an inertial position tracker and a depth somatosensory camera, characterized in that: include: Data acquisition module: used to collect the posture signals of the skeleton points of the human upper limbs through the depth sensing camera, and collect the posture signals of the human elbow joint through the inertial position tracker; Coordinate system construction module: used to construct a global coordinate system, and convert the upper limb skeleton bone point posture signal and the elbow joint posture signal into the global coordinate system; Specifically, a global coordinate system is constructed according to the human elbow joint posture signal collected by the inertial position tracker, and a depth somatosensory camera coordinate system is constructed according to the human upper limb skeleton bone point posture signal collected by the depth somatosensory camera, wherein the global coordinate system is parallel to the depth somatosensory camera coordinate system; and the upper limb skeleton bone point posture signal is converted into the global coordinate system according to the translation amount of the two coordinate systems; Data processing module: used for obtaining the positions of the elbow joint and the wrist joint in the global coordinate system according to the double Kalman filtering method, and calculating the elbow joint angle in combination with the shoulder joint position obtained according to the upper limb skeleton bone point posture signal; The method of obtaining the positions of the elbow joint and the wrist joint in the global coordinate system according to the dual Kalman filtering method includes: The elbow joint position is calculated using the Kalman filter method, specifically: establishing an elbow joint state equation according to the elbow joint posture signal, establishing an elbow joint observation equation according to the upper limb skeleton bone point posture signal, and introducing a flag indicating whether the elbow joint bone point is blocked in the upper limb skeleton bone point posture signal. , predict and update the elbow joint position; The wrist joint position is calculated using the Kalman filter method, specifically: the direction vector from the elbow to the wrist joint is established according to the predicted updated elbow joint position and the elbow joint posture signal, the direction vector state equation from the elbow to the wrist joint is established according to the elbow joint posture signal, the direction vector observation equation from the elbow to the wrist joint is established according to the upper limb skeleton bone point posture signal, and the wrist joint bone point is introduced in the upper limb skeleton bone point posture signal Whether the wrist joint bone point is blocked , predict and update the direction vector from the elbow joint to the wrist joint, and determine the wrist joint position according to the predicted and updated direction vector from the elbow joint to the wrist joint.

Images