The application discloses a kind of based on the dual hand
reconstruction method of multi-scale color information and depth
information fusion.Utilize dual-flow
feature extraction model respectively to extract
monocular RGB-D image multi-scale color feature, left and right hand corresponding center point feature and multi-scale depth feature;Then use
pyramid type depth fusion network to fuse multi-scale color feature, left and right hand corresponding center point feature and multi-scale depth feature, obtain left and right hand corresponding global fusion feature;Finally, according to left and right hand corresponding global fusion feature, using GCN decoder generates dual hand grid in camera coordinate
system.The application can fully fuse color information and depth information to reconstruct both hands, reduce the blur effect in
RGB image and the
noise in depth image, greatly improve the reconstruction accuracy, and enhance the robustness of model;Meanwhile, the application can accurately reconstruct three-dimensional hand grid with real depth and scale in camera space, overcome the limitation of relying on root node output.