Image rendering model training method and device and image rendering method and device

Abstract

The invention provides an image rendering model training method and device and an image rendering method and device. The model training method comprises the steps of determining a multi-angle target scene graph of a target scene; performing projection reconstruction on the multi-angle target scene graph to obtain an explicit density distribution matrix for representing the 3D scene density of the target scene; voxel sampling is carried out on the initial neural radiation field of the target scene, and an initial rendering image is generated based on voxel features containing density and color values in sampling points obtained through voxel sampling; and training the initial neural radiation field based on the difference between the initial rendering image and the multi-angle target scene graph and the difference between the density of the sampling points in each region and the density of the corresponding region in the explicit density distribution matrix to obtain an image rendering model. On the basis of not increasing additional forward operation, convergence of the loss function is accelerated, and the training speed of the model is improved.

Description

technical field [0001] The present invention relates to the technical field of image processing, in particular to an image rendering model training, image rendering method and device. Background technique [0002] NeRF (Neural Radiance Fields) is a deep rendering method whose main features are implicit representation of scenes and volumetric rendering of images. NeRF first learns the scene representation according to the images of different perspectives of the scene, and then generates images of any specified angle from the representation. [0003] After the scene representation is learned from the images of different viewing angles of the scene, to generate an image of any specified angle, it is necessary to sample the scene representation, and perform image rendering based on the sampled voxels to obtain the rendered image. [0004] Due to the uneven distribution of scene features in the volume generated based on NeRF, in order to improve the sampling efficiency, NeRF ado...

AI Technical Summary

Problems solved by technology

[0006] The present invention provides an image rendering model training, image rendering method and device, which are used to solve the defects of large amount of calculation and low efficiency of image rendering in the prior art

Images