Three-dimensional human body and object rendering method and application method thereof

Abstract

The invention provides a three-dimensional human body and object rendering method and an application method thereof, and the rendering method comprises the steps: carrying out the image collection through a plurality of calibrated cameras, and storing a collected color image, and the corresponding depth and camera visual angle; based on the acquired color image, acquiring a human body posture by using a human body posture estimation algorithm; driving the deformable structured human body three-dimensional model by using a human body posture to obtain a human body geometric appearance; optimizing a geometric learning multi-layer perceptron by using the geometric appearance of the human body, the color image and the corresponding depth and camera visual angle; using sampling points on the surface of the object to optimize colors to learn a multi-layer perceptron; and performing multi-view color image synthesis by using the optimized color learning multi-layer perceptron. According to the method, the rendering effect of the human body and the object under the sparse view angle can be improved by adopting the prior coding, the geometric learning multi-layer perceptron and the color learning multi-layer perceptron.

Description

technical field [0001] The application belongs to the technical field of image processing, and in particular relates to a rendering method of a three-dimensional human body and an object and an application method thereof. Background technique [0002] Whether it is a film and television program or a sports event, the images we see are all captured by a single camera. Under such conditions, people's viewing angle is firmly bound to the camera's angle of view. In order to get rid of these limitations, using any-view neural rendering technology, it only needs to input a small number of human body shots taken at angles to synthesize a 360° new 3D view of the human body, allowing the audience to obtain a fully immersive experience. [0003] However, existing neural rendering techniques do not generalize well to display applications. On the one hand, existing rendering techniques require very dense views to train view synthesis networks, generally requiring 100 cameras. To capt...

AI Technical Summary

Problems solved by technology

In order to capture dense multi-view video, traditional methods require expensive camera arrays to capture

This camera array equipment is very expensive and inconvenient, and it is generally only used in film shooting

On the other hand, the existing synthesis technology can only synthesize static scenes; for dynamic scenes, the existing technology can only treat a single frame as a static scene; 120 networks need to be trained to make predictions for each frame, and there is no connection between each frame of video, resulting in a huge amount of calculation and high complexity

Second, existing arbitrary view synthesis techniques require extremely high computing power

Therefore, existing rendering techniques cannot be applied to VR glasses

Images