NERF optimization method and device based on attention mechanism, equipment and readable medium

Abstract

The invention provides an attention mechanism-based NERF optimization method, device and equipment and a readable medium, and the method comprises the steps: obtaining a plurality of pictures shot at different positions in a 3D scene and information of the pictures, and setting parameters for processing pixels in the pictures by taking a pixel patch as a unit; processing the coordinate dimension of the light beam according to parameters, and performing primary coding on coordinate information of all pixel points in one patch by using an attention mechanism; uniformly sampling a plurality of sampling points in the light beam, and obtaining the RGB value of each sampling point in the plurality of sampling points according to the information of the coordinate dimension of the light beam; and carrying out volume rendering on the plurality of sampling points to synthesize a color RGB, selecting a fine sampling point based on the contribution probability of each point to the light beam color, and carrying out volume rendering synthesis on the fine sampling point to obtain a value of the color RGB. By using the scheme of the invention, the speed of model training and reasoning can be accelerated, and the rendering effect of the model can be improved.

Description

Technical field [0001] The invention involves the computer field, and more specifically involves a NERF optimization method, device, equipment and readable medium based on a attention mechanism. Background technique [0002] The rendering in computer graphics refers to the optical process of using a computer to simulate the actual physical world. When studying rendering technology, it also includes 3D scenario model construction and representation. Only when 3D scenarios are implemented in the computer can rendering be completed based on this. The currently widely used rendering technology is implemented based on computer graphics, such as grating, light projection, light / path tracking. Display in the screen. As the requirements for picture quality have increased, such as games, movies and other industries, algorithms such as light / path tracking are constantly accurate, and the industry also computes based on hardware such as GPU, but the problems between accuracy and efficiency ...

AI Technical Summary

Problems solved by technology

[0006] In the existing technology represented by AutoInt, the neural network is used to replace the process of calculating numerical integration, that is, the process of backpropagation is used to replace the value of the network input end of forward propagation to solve the problem, but it still cannot get rid of the problem of calculating many rays one by one. Taking a 1080P image as an example (1920*1080), 2 million rays need to be calculated; autoInt regards the projection of a ray in volumetric rendering as a definite integral, and defines the corresponding neural network G to represent the integration process, and then the neural network G Derivation to obtain the corresponding derivative network D

Images