Three-dimensional human body posture estimation method and device

Abstract

The invention discloses a three-dimensional human body posture estimation method and device. The method comprises the following steps of S1, using a monocular camera to collect the depth images and the RGB color images of a human body at different angles; S2, constructing a human body skeleton key point detection nerve network based on the RGB color images, and obtaining a key point labeling image; S3, constructing hand joint node 2D-3D mapping network; S4, calibrating the depth image of the human body at the same angle and the key point labeling image, and then carrying out three-dimensionalpoint cloud color conversion on the corresponding depth image so as to obtain a colored depth image; S5, based on the key point labeling image and the colored depth image, using a preset learning network to predict the corresponding position of the labeled human body skeleton key point in the depth image; and S6, combining the outputs of the step S3 and the step S5 to achieve the refined estimation of three-dimensional human body posture estimation.

Description

technical field [0001] The invention belongs to the application fields of computer vision, image processing, computer graphics and deep learning, and in particular relates to a three-dimensional human body pose estimation method and device. Background technique [0002] The so-called human pose estimation refers to matching the abstract hierarchical features with the human body model, so as to obtain the poses of the target at different times. Human pose estimation is the core problem of human motion capture. The posture expression of the human body includes two aspects, one is the position and direction of the whole human body in the world coordinates; the other is the angle of the joints of each part of the body and the skin deformation affected by the joint angles. The main application areas of human motion pose estimation can be divided into three major directions: monitoring, control and analysis: [0003] (1) In terms of monitoring applications, some traditional appl...

AI Technical Summary

Problems solved by technology

Magnetic or optical markers are attached to human limbs, and their three-dimensional trajectories are used to describe the target movement. These systems are automatic, but their disadvantages are: the equipment is very bulky and expensive, and cannot be widely used

[0011] The second is to use the part detection method to determine the various parts of the body after removing the background other than the human body. This method can be partially automated, but requires strict guarantees for human scene segmentation

The disadvantage of this method is that the tracking of subsequent frames has cumulative errors, and long-term tracking is prone to errors

The disadvantage of this method is that it is difficult to describe the human motion posture with a limited number of states, it depends on prior knowledge, and it can only track a specific set of actions.

However, the conditions for using multi-eye cameras are: calibration is required and it is inconvenient to arrange in ordinary households, which is not conducive to the popularization of motion capture technology to thousands of households

Images