Three-dimensional human body model joint reconstruction method based on monocular image, electronic equipment and storage medium

Abstract

The invention discloses a three-dimensional human body model joint reconstruction method based on a monocular image, electronic equipment and a storage medium, and the method comprises the steps: obtaining a body monocular image, a hand monocular image and a face monocular image based on an original monocular image, and extracting the coordinates of two-dimensional key points in the body monocular image, the hand monocular image and the face monocular image; the method comprises the following steps of: obtaining a body candidate frame, a hand candidate frame and a face candidate frame, cutting an original monocular image based on the three candidate frames to obtain three feature images, respectively obtaining features in each feature image by using a local feature extraction network, splicing the features to obtain cascade features, training an SMPLX model, and carrying out three-dimensional human body reconstruction based on the cascade features; the depth information extracted by the method is richer, and the constructed three-dimensional human body model is higher in precision and shorter in time consumption.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and in particular relates to a method for joint reconstruction of a three-dimensional human body model based on a monocular image, an electronic device and a storage medium. Background technique [0002] In recent years, billions of human daily activities have been recorded as videos and uploaded to public Internet sites to capture diverse human behaviors in various real-world scenarios. It has great potential in various applications including social artificial intelligence and robotics; 3D human reconstruction model methods based on monocular images have attracted a lot of attention, and there are problems such as lack of depth information in monocular image reconstruction. [0003] At present, the 3D human reconstruction method based on monocular image only includes the posture and body shape of the human body, ignoring the reconstruction of hand posture and facial expression, and in th...

AI Technical Summary

Problems solved by technology

[0002] In recent years, billions of human daily activities have been recorded as videos and uploaded to public Internet sites to capture diverse human behaviors in various real-world scenarios. The technology of digitizing human actions in these videos has been widely used in human-computer interaction, It has great potential in various applications including social artificial intelligence and robotics; 3D human body reconstruction model methods based on monocular images have attracted a lot of attention, and there are problems such as lack of depth information in monocular image reconstruction

[0003] The current 3D human body reconstruction method based on monocular images only includes the posture and body shape of the human body, ignoring the reconstruction of hand posture and facial expressions, and in the reconstruction of the overall 3D human body model, it is necessary to face the comparison of the proportion of the hand and face in the image. Small, difficult to capture accurate poses and other issues, which limit the accuracy of human body model reconstruction in pose capture systems, and restrict the computer from understanding human actions in a more intelligent way in human-computer interaction

Images