Image rendering model training method and device and image rendering method and device

Abstract

The invention provides an image rendering model training method and device and an image rendering method and device, and the model training method comprises the steps: inputting a multi-angle target scene graph into a volume rendering model, and obtaining a volume rendering image outputted by the volume rendering model; based on the volume rendering image and the multi-angle target scene graph, training the initialized neural radiation field with implicit scene expression ability to obtain an image rendering model, wherein the volume rendering model is obtained based on multi-angle sample scene graph training; enabling the volume rendering model to firstly carry out projection reconstruction on the multi-angle target scene graph to obtain an explicit density distribution matrix used for representing 3D scene density of a target scene, and after sampling points in the projection direction are determined based on the density distribution matrix, generating a volume rendering image based on voxel features including density and color values in the sampling points. According to the method and device, the sampling points can be quickly and directly determined based on the explicit density distribution matrix, and the training and reasoning efficiency of the image rendering model is improved.

Description

technical field [0001] The present invention relates to the technical field of image processing, in particular to an image rendering model training, image rendering method and device. Background technique [0002] NeRF (Neural Radiance Fields) is a deep rendering method whose main features are implicit representation of scenes and volumetric rendering of images. NeRF first learns the scene representation according to the images of different perspectives of the scene, and then generates images of any specified angle from the representation. [0003] After the scene representation is learned from the images of different viewing angles of the scene, to generate an image of any specified angle, it is necessary to sample the scene representation, and perform image rendering based on the sampled voxels to obtain the rendered image. [0004] Due to the uneven distribution of scene features in the volume generated based on NeRF, in order to improve the sampling efficiency, NeRF ado...

AI Technical Summary

Problems solved by technology

[0006] The present invention provides an image rendering model training, image rendering method and device, which are used to solve the defects of large amount of calculation and low efficiency of image rendering in the prior art

Images