Method and device for optimizing human body depth map through neural network

Abstract

The invention discloses a method and device for optimizing a human body depth map through a neural network. A deep neural network adopted by the method takes a rough depth map and a color image as input, and takes an accurate depth map as output; in the training stage, fusion optimization is carried out on multiple continuous frames of depth maps to obtain depth maps with high precision and colorpictures aligned with the depth maps as training data; and through training, a neural network model capable of optimizing the rough depth map is obtained. The device comprises an image sequence and model acquisition module, a model and image sequence alignment module, a human body model extraction module, a network data preprocessing module, a network design module and a network training and prediction module. The depth map predicted by the method has lower noise and smoother appearance, and has higher precision compared with an input rough depth map.

Description

technical field [0001] The invention relates to the field of computer vision, in particular to a method and a device for optimizing a human body depth map through a neural network. Background technique [0002] In recent years, with the development of computer vision and virtual reality technology, the value of 3D reconstruction (3D reconstruction) has gradually become prominent in many fields such as film and television production, virtual reality, game production, and modern medical care. How to optimize to obtain a smooth, low-noise 3D Modeling becomes a widespread concern. [0003] The proposed algorithms for optimizing human body depth maps can be roughly divided into two categories: [0004] The first type of methods enhance depth data through RGB images, usually make some heuristic assumptions about the correlation between color and depth, and then use RGB information to optimize the original depth data. Such methods are based on the heuristic assumption between col...

AI Technical Summary

Problems solved by technology

However, the traditional method often needs to fuse multiple frames to suppress noise. Although a lot of progress has been made in reducing noise and enhancing geometric details, it still cannot meet the requirements of real-time performance.

At the same time, the traditional method performs time-domain filtering on the original depth to reduce sensor noise, but simple time-domain filtering will smooth the high-frequency structure, resulting in irreparable detail degradation, which cannot accurately represent the fine 3D shape

Images