Construction method of multi-resolution light field model

Abstract

The invention discloses a construction method of a multi-resolution light field model, and relates to the field of computer graphics and three-dimensional modeling. The method comprises the following steps: firstly, sampling a neural radiation field to obtain a dense grid model; then, constructing a multi-resolution light field model of a sparse octree structure according to the dense grid model; and finally, carrying out data compression on the sparse octree nodes. According to the method, the light field model supporting multi-resolution expression can be efficiently constructed, the storage capacity, the rendering quality and the rendering efficiency of the light field model are considered, the real-time rendering requirement can be met, the overall performance and efficiency of multi-resolution visual expression of the model are improved, and the burden of processing, transmission and rendering of scenes of different scales is effectively relieved.

Description

Technical Field

[0001] The present invention relates to the fields of computer graphics and three-dimensional modeling, and in particular to a method for constructing a multi-resolution light field model. Background Art

[0002] In the fields of urban planning, engineering surveying, medical care, virtual reality, etc., three-dimensional models are digital expressions of real-world objects, which can bring users a more realistic virtual experience and establish a connection between the real world and the virtual world.

[0003] Explicit expression is the most commonly used method for expressing three-dimensional models, but it is difficult to accurately express complex structures. In addition, the commonly used local illumination model is essentially a form of attribute interpolation and can only approximately express the real scene. Although the global illumination model can express complex light effects, the surge in computational complexity leads to a sharp decline in rendering performance.

[0004] Implicit representations are not limited by spatial resolution and possess stronger representational capabilities. Neural Radiance Fields (NeRF), a recently emerging technique, is essentially an implicit representation. It can capture and simulate spatial light field information based on line of sight, generating highly realistic 3D scenes. However, generalized NeRF requires frequent neural network queries, making it difficult to meet the demands of real-time rendering.

[0005] Taking NeRF offline, pre-calculating and structured storing scene geometry and radiosity, and thus eliminating reliance on neural networks, is an effective way to overcome inference bottlenecks and improve rendering performance. Existing research uses sparse voxel grids to organize offline data, which reduces video memory usage but still faces the dilemma of redundant storage and loss of detail in complex scenes.

[0006] Therefore, how to achieve a balance among storage capacity, rendering quality and rendering efficiency is the key issue that NeRF offline needs to solve. Summary of the Invention

[0007] The problem to be solved by the present invention is to provide a method for constructing a multi-resolution light field model. While ensuring the rendering quality, the method solves the real-time rendering bottleneck problem of the generalized neural radiation field model by reorganizing the scene data. The method uses multi-resolution technology to balance the data volume and rendering speed, thereby alleviating the performance pressure brought by the rendering of scene models of different scales.

[0008] The present invention adopts the following technical solution: a method for constructing a multi-resolution light field model, comprising the following steps:

[0009] S1. Sample the neural radiation field and build a dense grid model;

[0010] S2. Based on the dense grid model, a multi-resolution light field model based on a sparse octree structure is constructed;

[0011] S3. Quantize the color of the sparse octree nodes using a radix sorting algorithm to reduce storage space.

[0012] Preferably, the light field model is a discretized neural radiation field model, which uses density and spherical harmonic coefficients to record the irradiance information of spatial points; the multi-resolution light field model is a light field model that supports multi-resolution expression.

[0013] Preferably, step S1 samples the neural radiation field and establishes a dense grid model, including the following sub-steps:

[0014] S1.1. Determine the three-dimensional range of the dense grid model based on the scene covered by the neural radiation field, and then determine the sampling side length SIZE (integer power of 2) of the dense grid unit and the total number of sampling points N based on the number of sampling points N in the X, Y, and Z directions. 3 , forming a dense sampling grid;

[0015] S1.2. Sample the neural radiation field according to the three-dimensional coordinates of each sampling grid point to obtain the density value and spherical harmonic coefficient as the attribute value of the dense sampling grid node.

[0016] Preferably, step S2 constructs a multi-resolution light field model with a sparse octree structure based on the dense grid model, including the following sub-steps:

[0017] S2.1. Build the highest resolution layer:

[0018] Determine the minimum density threshold σ for recursive termination when constructing a sparse octree, recursively split the current node into 8 child nodes, and the recursive termination condition is: the edge length of the current node is equal to SIZE or the density values ​​of the sampling points covered by the current node are all less than σ; the leaf node stores the density values ​​of all sampling points set C covered by the node and the downsampling results of the spherical harmonic coefficients;

[0019] S2.2. Construct other resolution layers:

[0020] Other resolution light field data are generated from the bottom up and stored in non-leaf nodes, specifically the parent node stores the downsampled data of the child node.

[0021] Preferably, in step S3, the sparse octree node colors are quantized into a color space represented by spherical harmonic coefficients by a radix sorting algorithm. order colors, where order represents the number of color bit planes, including the following sub-steps:

[0022] S3.1, build a color mapping table, the size of which is 2 order,The color map element is used to store the complete spherical harmonic coefficients of the node;

[0023] S3.2. For all nodes, the spherical harmonic coefficients are sorted in cardinality order from high to low order;

[0024] S3.3, divide the sorted spherical harmonic coefficients into 2 order The geometric mean of the spherical harmonic coefficients of each group is taken, and then stored in the corresponding position of the color mapping table according to the group number;

[0025] S3.4. The node color is represented by the group number instead of the spherical harmonic coefficient to realize color quantization coding.

[0026] The technical solution of the present invention further provides: an electronic device, comprising:

[0027] one or more processors;

[0028] a storage device having one or more programs stored thereon;

[0029] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the above-mentioned methods for constructing a multi-resolution light field model.

[0030] The technical solution of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps in any of the above-mentioned methods for constructing a multi-resolution light field model are implemented.

[0031] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0032] The present invention designs a universal, efficient, and compact light field model data structure, which realizes the efficient organization of scene data of different scales and supports the multi-resolution expression of light field models. Through color quantization encoding, it takes into account the storage capacity, rendering quality, and rendering efficiency of the light field model, can meet the needs of real-time rendering, and realize more efficient and flexible scene visualization. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart for constructing a multi-resolution light field model of the present invention;

[0034] Figure 2 This is a sparse octree structure diagram of the present invention;

[0035] Figure 3 This is a comparison chart of compression effects of an embodiment of the present invention; DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the application are further elaborated in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in the present invention. All non-innovative embodiments of other researchers in this field on this embodiment fall within the scope of protection of the present invention. At the same time, the step numbers in the embodiments of the present invention are only set for the convenience of explanation and description, and do not limit the order between the steps. The execution order of each step in the embodiment can be adaptively adjusted according to the understanding of those skilled in the art.

[0037] The present invention provides a method for constructing a multi-resolution light field model. The light field model is a discretized neural radiation field model, which uses density and spherical harmonic coefficients to record the irradiance information of spatial points. The multi-resolution light field model is a light field model that supports multi-resolution expression.

[0038] In one embodiment of the present invention, the experimental environment is divided into two parts: a training server and a local platform. The experimental data is an MP4 format video shot by the DJI Mavic 3 series drone. The video length is 1 minute 20 seconds and the frame rate is 30fps.

[0039] Based on the experimental data, the multi-resolution light field model of the present invention is constructed. The method flow is as follows: Figure 1 The specific steps are as follows:

[0040] S1. Sample the neural radiation field and establish a dense grid model;

[0041] S1.1. Determine the three-dimensional range of the dense grid model based on the scene covered by the neural radiation field, and then determine the sampling side length SIZE of the dense grid unit and the total number of sampling grid points N based on the number of sampling points N in the X, Y, and Z directions (N is an integer power of 2). 3 .

[0042] S1.2. Sample the neural radiation field according to the three-dimensional coordinates of the grid points to obtain the density value and spherical harmonic coefficient of the sampling points as the attribute value of the dense grid node.

[0043] In this embodiment, N is selected as 512, 256, 128, 64, 32, and 16 for grid resampling. The sampling function is defined as a thread entry function, and multi-threading technology is used to process the discrete point set in batches in parallel to calculate the density value and spherical harmonic coefficient of each sampling point.

[0044] S2. Construct a multi-resolution light field model with a sparse octree structure based on the dense grid model;

[0045] A sparse octree structure is constructed to organize multi-resolution light field data, and light field data of different resolutions are stored at different levels of the tree.

[0046] S2.1. Build the highest resolution layer.

[0047] In this embodiment, the minimum density threshold σ for recursive termination when constructing a sparse octree is determined to be 0.1, and the current node is recursively split into 8 child nodes. The recursive termination condition is: the side length of the current node is equal to SIZE or the density values ​​of the sampling points covered by the current node are all less than σ.

[0048] The leaf node stores the downsampling results of the density values ​​and spherical harmonic coefficients of all sampling point sets C covered by the node. In this embodiment, the specific operation of downsampling is to obtain the data of the first member of C.

[0049] S2.2. Construct other resolution layers.

[0050] Generate other resolution data from the bottom up and store them in non-leaf nodes. Specifically, the parent node stores the downsampled data of the child node, and finally builds a hierarchical sparse octree structure, such as Figure 2 shown.

[0051] S3. Quantize the color of the sparse octree nodes using a radix sorting algorithm to reduce storage space.

[0052] In this embodiment, a radix sorting algorithm is used to quantize the color space represented by the spherical harmonic coefficients into 2 16 colors, where 16 represents the number of color bit planes, specifically including the following sub-steps:

[0053] S3.1, build a color mapping table, the size of which is 2 16 ,The color map element is used to store the complete spherical harmonic coefficients of the node;

[0054] S3.2. For all nodes, the spherical harmonic coefficients are sorted in cardinality order from high to low order;

[0055] S3.3, divide the sorted spherical harmonic coefficients into 2 16 The geometric mean of the spherical harmonic coefficients of each group is taken, and then stored in the corresponding position of the color mapping table according to the group number;

[0056] S3.4. The node color is represented by group number instead of spherical harmonic coefficient to realize color quantization coding. The quantization compression effect is compared. Figure 3 shown.

[0057] In an embodiment of the present invention, an electronic device is also provided, comprising: one or more processors; a storage device having one or more programs stored thereon; when the one or more programs are executed by the one or more processors, the one or more processors implement the method for constructing a multi-resolution light field model described in any of the above embodiments.

[0058] In an embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored. When the program is executed by a processor, the steps of the method for constructing a multi-resolution light field model in any of the above embodiments are implemented.

[0059] In summary, the multi-resolution light field model construction method of the present invention can efficiently construct a light field model that supports multi-resolution expression, improve the overall performance and efficiency of the model's multi-resolution visualization expression, and effectively alleviate the burden of processing, transmitting, and rendering scenes of different scales.

[0060] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for constructing a multi-resolution light field model, characterized in that: The steps include: S1. Sample the neural radiation field and establish a dense grid model; S2. Based on the dense grid model, a multi-resolution light field model based on a sparse octree structure is constructed; S3. Quantize the color of the sparse octree nodes using a radix sorting algorithm to reduce storage space.

2. The method for constructing a multi-resolution light field model according to claim 1, wherein: The light field model is a discretized neural radiation field model that uses density and spherical harmonic coefficients to record the irradiance information of spatial points; the multi-resolution light field model is a light field model that supports multi-resolution expression.

3. The method for constructing a multi-resolution light field model according to claim 1, wherein: In step S1, a dense grid model is established, which includes the following sub-steps: S1.

1. Determine the three-dimensional range of the dense grid model based on the scene covered by the neural radiation field, and determine the sampling side length SIZE of the dense grid unit and the total number of sampling grid points N based on the number of sampling points N in the X, Y, and Z directions. 3 , forming a dense sampling grid; S1.

2. Sample the neural radiation field according to the three-dimensional coordinates of each sampling grid point to obtain the density value and spherical harmonic coefficient as the attribute value of the dense sampling grid point.

4. The method for constructing a multi-resolution light field model according to claim 3, wherein: In step S1.1, the number of sampling points N is an integer power of 2, and the sampling grid points are discrete points.

5. The method for constructing a multi-resolution light field model according to claim 4, wherein: In step S2, the sparse octree structure is used to organize multi-resolution light field data, and light field data of different resolutions are stored in different levels of the tree.

6. The method for constructing a multi-resolution light field model according to claim 5, wherein: In step S2, a multi-resolution light field model based on a sparse octree structure is constructed, including the following sub-steps: S2.

1. Build the highest resolution layer: Determine the minimum density threshold σ for recursive termination when constructing a sparse octree, recursively split the current node into 8 child nodes, and the recursive termination conditions are: the edge length of the current node is equal to SIZE or the density values ​​of all sampling points covered by the current node are less than σ. The leaf node stores the density values ​​of all sampling point sets C covered by the node and the downsampling results of the spherical harmonic coefficients; The downsampling strategy is: take the data of the first member of C, or the data of a randomly selected member of C, or take the mean of the data of all members of C; S2.

2. Construct other resolution layers: Other resolution light field data are generated from the bottom up and stored in non-leaf nodes, specifically the parent node stores the downsampled data of the child node.

7. The method for constructing a multi-resolution light field model according to claim 5, wherein: In step S3, the radix sorting algorithm is used to quantize the color of the sparse octree nodes, and the color space represented by the spherical harmonic coefficients is quantized to 2 order colors, where order represents the number of color bit planes, and specifically includes the following sub-steps: S3.1, build a color mapping table, the size of which is 2 order ,The color map element is used to store the complete spherical harmonic coefficients of the node; S3.

2. For all nodes, the spherical harmonic coefficients are sorted in cardinality order from high to low order; S3.3, divide the sorted spherical harmonic coefficients into 2 order The geometric mean of the spherical harmonic coefficients of each group is taken, and then stored in the corresponding position of the color mapping table according to the group number; S3.

4. The node color is represented by the group number instead of the spherical harmonic coefficient to realize color quantization coding.

8. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for constructing a multi-resolution light field model as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the program is executed by a processor, the steps in the method for constructing a multi-resolution light field model according to any one of claims 1 to 7 are implemented.

Images