NeRF Modeling Method Based on Multi-Scale Voxel Fusion and Total Variation Regularization

Abstract

This invention discloses a NeRF modeling method based on multi-scale voxel fusion and total variational regularization, belonging to the field of image data processing. The method includes the following steps: constructing a dataset D; generating sampling points; generating L layers of spatial features for a forestry 3D scene; for any sampling point x, generating its multi-scale features in each layer of spatial features; constructing a multi-scale fusion NeRF network to generate input vectors based on the multi-scale features, and then using an MLP to output predicted volume density and predicted color; constructing a total loss L; training the multi-scale fusion NeRF network to obtain a forestry 3D scene model, which is used for reconstructing new perspective images in forestry 3D scenes. This invention improves the detail restoration and global consistency of 3D reconstruction, ensures the spatial structure stability and geometric continuity in dynamic scenes, and solves problems such as high reconstruction noise, loss of detail, and poor adaptability to dynamic scenes. It is suitable for high-precision 3D scene modeling and real-time ecological monitoring.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a NeRF modeling method based on multi-scale voxel fusion and total variational regularization. Background Technology

[0002] With the rapid development of 3D reconstruction and intelligent visual recognition technologies, neural radiation field (NeRF)-based methods have shown great potential in fields such as forestry digital twins, environmental monitoring, and virtual reality. In the field of forestry 3D modeling, ensuring high-quality point clouds and voxel reconstruction is crucial for tasks such as tree identification, structural analysis, and stand evaluation. Practical applications often require a high degree of control over detail richness, geometric accuracy, and dynamic adaptability to meet objectives such as forestry resource surveys and automated monitoring. However, the following problems exist:

[0003] (1) Existing methods are difficult to balance detailed representation and global consistency. For example, traditional NeRF is difficult to simultaneously take into account coarse-scale structures such as tree trunks and branches and fine-scale features such as leaves and textures. In forest scenes with strong coupling of multi-scale structures, problems such as blurred textures, broken branches or missing details are likely to occur.

[0004] (2) The reconstruction is susceptible to noise and lacks adaptability to dynamic scenes. This is because the reconstruction process often relies on local light field fitting and lacks constraints on the global structure, which may cause distortion of the generated 3D structure in a large area, discontinuous tree shape, and even structural drift between multiple frames. Summary of the Invention

[0005] The purpose of this invention is to provide a NeRF modeling method based on multi-scale voxel fusion and total variational regularization that solves the above problems, not only significantly improves the detail restoration and global consistency of 3D reconstruction, but also ensures the spatial structural stability and geometric continuity in dynamic scenes, thereby promoting the development of applications such as intelligent analysis and ecological assessment of forestry scenes.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a NeRF modeling method based on multi-scale voxel fusion and total variation regularization, comprising the following steps;

[0007] S1, construct dataset D;

[0008] Use a camera to acquire N images of a 3D forestry scene from various perspectives, and then use the i-th image I... i Compared to the shooting timestamp t i Construct the i-th sample (I) i ,t i All samples constitute dataset D, I i From the camera origin o iThe ray pointing to pixel p is r i,p The viewing direction is d i,p ;

[0009] S2, performs coarse and fine sampling on each ray to generate N. c One coarse sampling point and N f Each fine sampling point;

[0010] S3, using L voxels with progressively decreasing resolutions, divide the forestry 3D scene into L 3D meshes, extract voxel features from each mesh, and use all voxel features corresponding to the l-th resolution voxel as the l-th layer spatial feature V of the forestry 3D scene. (l) , 1≤l≤L;

[0011] S4, for any sampling point x, generate its value in V. (l) Multiscale features are The sampling point x is either a coarse sampling point or a fine sampling point;

[0012] S5. Construct a multi-scale fusion NeRF network, including FPN-PAN multi-scale fusion unit, vector generation unit and MLP;

[0013] The FPN-PAN multi-scale fusion unit includes an FPN, a PAN, and a fusion layer;

[0014] For sampling point x, features are extracted from the bottom up by FPN, and the FPN fusion feature of x at the l-th layer is generated. Then, features are extracted from the PAN from top to bottom, and the features from the (l-1)th layer of the PAN are fused in the l-th layer. and Aggregation yields the PAN fusion feature of x at layer l. The fusion layer is used to combine L elements. Channel-by-channel concatenation yields the global feature v of x. multi (x);

[0015] The vector generation unit is used to generate the input vector at time t for each sampling point, where the input vector at time t for sampling point x is f(x,t). γ(x), γ(d), and γ(t) are the location of the sampling point x, the viewing direction of the ray, and the code corresponding to the shooting timestamp, respectively.

[0016] The MLP is used as input to f(x,t) and outputs the predicted volume density and predicted color of x;

[0017] S6, the total loss L of constructing a multi-scale fused NeRF network;

[0018] ,

[0019] In the formula, L recon For the MSE loss of true color and predicted color, L TV For three-dimensional total variational regularized loss, L time For time consistency loss, λ TV , λ time L respectively TV and L time The weights;

[0020] S7 uses dataset D to train a multi-scale fusion NeRF network by minimizing L, resulting in a forestry 3D scene model for reconstructing new perspective images in forestry 3D scenes.

[0021] As a preferred option, S2 is specifically as follows:

[0022] S21, in the ray r i,p Upper equal interval sampling N c The j-th coarse sample point is predicted by MLP from the given coarse sample points. volume density and color And calculate opacity and cumulative transmittance , 1≤j≤N c ;

[0023] ,

[0024] ,

[0025] In the formula, exp(⋅) is the exp function. for With the next coarse sampling point The step size between, The opacity of the k-th coarse sampling point;

[0026] S22, calculate the j-th coarse sampling point according to the following formula. sampling weights ;

[0027] ,

[0028] S23, will After normalization, the cumulative distribution function (CDF) is formed. Based on the CDF, N samples are uniformly sampled on [0,1]. f N random numbers are obtained by table lookup and interpolation. f Each fine sampling point.

[0029] Preferably, in S3, a learnable 3D convolutional network is used to extract the voxel features of each 3D grid to obtain V. (l) .

[0030] Preferably, in S4, the sampling point x is generated in V. (l) Multiscale features in The method is as follows:

[0031] In each layer V (l) The voxels to which the voxels belong are retrieved and multi-scale features are obtained by trilinear interpolation. ;

[0032] ,

[0033] In the formula, trilinear-interp(⋅) is the trilinear interpolation function.

[0034] Preferably, in S5, the FPN is used to generate the FPN fusion feature of x at layer l according to the following formula. ;

[0035] ,

[0036] The PAN is used to generate the PAN fusion feature of x in the l-th layer according to the following formula. ;

[0037] ,

[0038] The fusion layer is used to generate global features of x according to the following formula. ;

[0039] ,

[0040] In the formula, UpConv(∙) is the upsampling convolution operation. The FPN fusion feature for sampling point x at layer l+1 is used, and DownConv(∙) is the downsampling convolution operation. For the PAN fusion feature of sampling point x in layer l-1, Concat(∙) is the channel splicing operation; when l=L, When l=1, = .

[0041] As a preferred option, the three-dimensional total variational regularized loss L... TV Time consistency loss L time Calculate according to the following formulas respectively:

[0042] ,

[0043] In the formula, i, j, and k represent the spatial features V of the Lth layer, respectively. (L) The position index of a voxel in row, column, and height. , , , V (L) Voxel features at positions (i,j,k), (i+1,j,k), (i,j+1,k), and (i,j,k+1);

[0044] ,

[0045] In the formula, f(x,t+Δt) is the input vector of the sampling point at time t+Δt, MLP θ (∙) represents the predicted output of the MLP corresponding to the input vector, and Δt represents the time difference between the timestamps of adjacent image captures. It is the square of the L2 norm.

[0046] As a preferred option, the training in S7 is specifically as follows:

[0047] A ray is randomly selected from dataset D, and coarse and fine sampling points are generated on the ray. For any sampling point, an input vector is generated according to step S5, and the vector is fed into the MLP to obtain the predicted volume density and predicted color. The total loss L is calculated, and the parameters of the multi-scale fusion NeRF network are adjusted by minimizing L.

[0048] As a preferred method, the reconstruction of the new perspective image in S7 is as follows:

[0049] Generate rays from the camera origin to each pixel in the new viewpoint image;

[0050] Select any ray and generate sampling points on it. Pass the sampling points through the FPN-PAN multi-scale fusion unit and vector generation unit of the forestry 3D scene model to obtain the corresponding input vector. Then, pass the MLP to obtain the predicted volume density and predicted color, and calculate the color of the ray.

[0051] Each ray is processed sequentially to obtain its color, resulting in a new perspective image.

[0052] Regarding steps S1 and S2: In order to obtain the spatial information of the forestry 3D scene, it is first necessary to collect multiple sets of high-resolution images from different perspectives and time points. In this invention, each image I... i All data include the camera origin. i Camera facing d i Shooting timestamp t i Information such as the camera origin o i and the camera is facing d i We can get I i Zhongyouo i The ray pointing to pixel p is r i,p The viewing direction is d i,p Thus, in r i,p Upsampling.

[0053] Regarding step S3: To improve the modeling capability of multi-level structures in forestry scenes, this invention performs spatial multi-scale feature extraction on the 3D forestry scene through step S3. The 3D forestry scene is divided into 3D meshes using voxels of different resolutions, resulting in 3D meshes of varying resolutions. The first division uses the voxel resolution with the smallest voxel scale, and all voxel features contained in the 3D mesh under this division are labeled as V. (1) The voxel resolution decreases with each subsequent partitioning, with the Lth partition using the smallest voxel resolution and the largest voxel scale. All voxel features contained in the 3D mesh under this partition are denoted as V. (L) This yields the spatial features V at different resolutions in layer L. (1) ~V (L) It can be written as a whole. , 1≤l≤L.

[0054] Regarding step S4: The purpose is to obtain the spatial features V of any sampling point x in each layer. (l) Corresponding multi-scale features This is used for subsequent multi-scale fusion, and multiple different features It can meet the high requirements for target detail richness, geometric accuracy and dynamic adaptability in forestry resource surveys and automated monitoring.

[0055] Regarding step S5: this involves constructing a multi-scale fusion NeRF network to achieve... The network, which integrates multi-scale voxel features, comprises an FPN-PAN multi-scale fusion unit, a vector generation unit, and an MLP. The FPN-PAN multi-scale fusion unit includes an FPN, a PAN, and a fusion layer. The FPN extracts features from sampling point x from the bottom up, essentially upsampling the low-resolution (large voxels, coarse information) 3D feature map to a high-resolution (small voxels, fine information) feature map, allowing the abstract information from the low-resolution layer to supplement the high-resolution layer. The PAN, introduced from the top down, fully utilizes information from both shallow and deep layers. Essentially, it downsamples the high-resolution 3D voxel features to obtain low-resolution (larger voxels, abstract information) features, allowing the detailed high-resolution information to converge to the low-resolution layer through downsampling, supplementing the global abstraction capability. The fusion layer then integrates L features from different scales... Channel-by-channel concatenation yields the global feature v of x. multi (x). The vector generation unit is used to combine the spatial, viewpoint, and temporal information of each sampling point with multi-scale v. multi (x) are concatenated to form the input vector f(x,t) of the MLP. The MLP then predicts the volume density and color of x based on f(x,t) and outputs the predicted volume density and predicted color.

[0056] Regarding step S6: The total loss L includes L recon L TV L time Three parts. Among them, L... recon For the MSE loss of true color and predicted color, L TV This is a three-dimensional total variation regularized loss, which borrows the idea of ​​two-dimensional image total variation (TV) denoising and extends it to a three-dimensional voxel density field; L time The temporal consistency loss suppresses drastic changes between voxels, improves the smoothness and spatial consistency of the reconstruction results, prevents noise and artifacts, and ensures that the same spatial point appears smoothly in consecutive frames, making it suitable for dynamic forestry scenarios.

[0057] Regarding step S7: During the training phase, rays are randomly selected, and coarse and fine sampling points are obtained through layered sampling as in S2. Multi-scale features of the sampling points are generated according to S3 and S4. Feature fusion is performed in S5 to generate the input vector f(x,t) for the MLP, which is then fed into the MLP to predict volume density and color. Volume rendering integration yields the pixel color. Using L... recon +L TV +L time End-to-end optimization of all parameters. During the inference phase, at any new perspective and time, sampling, multi-scale features, and input vectors are generated according to the above S2~S5 process. Then, the volume density and color are inferred by MLP, and the new view is output by volume rendering.

[0058] Compared with the prior art, the advantages of the present invention are as follows:

[0059] (1) To meet the needs of 3D reconstruction and dynamic modeling of forestry scenes, the 3D forestry scene is divided into voxels of different scales to generate L layers of spatial features V at different resolutions. (1) ~V (L) This allows us to obtain multi-scale features corresponding to sampling points in different spatial features, which facilitates the extraction and fusion of spatial information at different scales and avoids the omission of local structures or the break in the overall modeling.

[0060] (2) Based on the generated L-layer spatial features V (1) ~V (L) For each sampling point, generate its corresponding value in each layer V. (l)Based on the multi-scale features in the model, an FPN-PAN multi-scale fusion unit is designed to extract and fuse multi-scale features according to the method of this invention. Through this FPN-PAN multi-scale fusion unit, this invention can simultaneously capture the global geometric structure and local details of the scene, achieving effective fusion of spatial information at different scales. This not only improves the detail resolution of the overall modeling but also ensures the consistency of elements at different scales, such as branches and canopies. It provides stronger data support for forestry 3D point cloud and voxel modeling, effectively reducing problems such as high noise, loss of detail, and poor adaptability to dynamic scenes in subsequent reconstructions. Furthermore, because it achieves effective fusion of spatial information at different scales, it can effectively avoid the defects of limited detail expression and global consistency when the model is dealing with large-scale or structurally complex scenes, which can easily lead to the omission of local structures or the fragmentation of the overall modeling.

[0061] (3) In the loss function, a three-dimensional total variational regular loss L is introduced. TV As a crucial step in the generation process, this method enhances the robustness of volume reconstruction to spatial noise and local anomalies. It not only focuses on the smoothness of the overall structure but also precisely suppresses subtle spatial variations and anomalies, significantly improving the practical effectiveness of forestry 3D modeling. Particularly in applications such as dynamic forest stand monitoring and tree structure restoration, this technology significantly enhances its engineering value.

[0062] In summary, this invention, by introducing multi-scale spatial feature fusion and three-dimensional total variational regularization, not only significantly improves the detail restoration and global consistency of 3D reconstruction, but also ensures the spatial structural stability and geometric continuity in dynamic scenes. This greatly enhances the quality and application value of forestry 3D modeling and intelligent reconstruction, providing an innovative technical path for forestry digital twins and 3D ecological scene perception. By addressing the problems of high reconstruction noise, detail loss, and poor adaptability to dynamic scenes in existing methods, this approach offers a new solution for specific tasks such as 3D management and automatic analysis of forestry resources, and has broad application prospects, particularly in high-precision 3D scene modeling and real-time ecological monitoring. Attached Figure Description

[0063] Figure 1 This is a flowchart of the present invention;

[0064] Figure 2 A diagram of the multi-scale fusion NeRF network structure;

[0065] Figure 3 This is a training flowchart. Detailed Implementation

[0066] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0067] Example 1: See Figures 1 to 3 A NeRF modeling method based on multi-scale voxel fusion and total variation regularization includes the following steps;

[0068] S1, construct dataset D;

[0069] Use a camera to acquire N images of a 3D forestry scene from various perspectives, and then use the i-th image I... i Compared to the shooting timestamp t i Construct the i-th sample (I) i ,t i All samples constitute dataset D, I i From the camera origin o i The ray pointing to pixel p is r i,p The viewing direction is d i,p ;

[0070] S2, performs coarse and fine sampling on each ray to generate N. c One coarse sampling point and N f Each fine sampling point;

[0071] S3, using L voxels with progressively decreasing resolutions, divide the forestry 3D scene into L 3D meshes, extract voxel features from each mesh, and use all voxel features corresponding to the l-th resolution voxel as the l-th layer spatial feature V of the forestry 3D scene. (l) , 1≤l≤L;

[0072] S4, for any sampling point x, generate its value in V. (l) Multiscale features are The sampling point x is either a coarse sampling point or a fine sampling point;

[0073] S5. Construct a multi-scale fusion NeRF network, including FPN-PAN multi-scale fusion unit, vector generation unit and MLP;

[0074] The FPN-PAN multi-scale fusion unit includes an FPN, a PAN, and a fusion layer;

[0075] For sampling point x, features are extracted from the bottom up by FPN, and the FPN fusion feature of x at the l-th layer is generated. Then, features are extracted from the PAN from top to bottom, and the features from the (l-1)th layer of the PAN are fused in the l-th layer. and Aggregation yields the PAN fusion feature of x at layer l. The fusion layer is used to combine L elements. Channel-by-channel concatenation yields the global feature v of x. multi (x);

[0076] The vector generation unit is used to generate the input vector at time t for each sampling point, where the input vector at time t for sampling point x is f(x,t). γ(x), γ(d), and γ(t) are the location of the sampling point x, the viewing direction of the ray, and the code corresponding to the shooting timestamp, respectively.

[0077] The MLP is used as input to f(x,t) and outputs the predicted volume density and predicted color of x;

[0078] S6, the total loss L of constructing a multi-scale fused NeRF network;

[0079] ,

[0080] In the formula, L recon For the MSE loss of true color and predicted color, L TV For three-dimensional total variational regularized loss, L time For time consistency loss, λ TV , λ time L respectively TV and L time The weights;

[0081] S7 uses dataset D to train a multi-scale fusion NeRF network by minimizing L, resulting in a forestry 3D scene model for reconstructing new perspective images in forestry 3D scenes.

[0082] Example 2: See Figure 1 and Figure 3 Based on Example 1, S2 specifically includes:

[0083] S21, in the ray r i,p Upper equal interval sampling N c The j-th coarse sample point is predicted by MLP from the given coarse sample points. volume density and color And calculate opacity and cumulative transmittance , 1≤j≤N c ;

[0084] ,

[0085] ,

[0086] In the formula, exp(⋅) is the exp function. for With the next coarse sampling point The step size between, The opacity of the k-th coarse sampling point;

[0087] S22, calculate the j-th coarse sampling point according to the following formula. sampling weights ;

[0088] ,

[0089] S23, will After normalization, the cumulative distribution function (CDF) is formed. Based on the CDF, N samples are uniformly sampled on [0,1]. f N random numbers are obtained by table lookup and interpolation. f Each fine sampling point.

[0090] In S3, a learnable 3D convolutional network is used to extract the voxel features of each 3D grid to obtain V. (l) .

[0091] In S4, the generated sampling point x is in V (l) Multiscale features in The method is as follows:

[0092] In each layer V (l) The voxels to which the voxels belong are retrieved and multi-scale features are obtained by trilinear interpolation. ;

[0093] ,

[0094] In the formula, trilinear-interp(⋅) is the trilinear interpolation function.

[0095] In S5, the FPN is used to generate the FPN fusion feature of x at layer l according to the following formula. ;

[0096] ,

[0097] The PAN is used to generate the PAN fusion feature of x in the l-th layer according to the following formula. ;

[0098] ,

[0099] The fusion layer is used to generate global features of x according to the following formula. ;

[0100] ,

[0101] In the formula, UpConv(∙) is the upsampling convolution operation. The FPN fusion feature for sampling point x at layer l+1 is used, and DownConv(∙) is the downsampling convolution operation. For the PAN fusion feature of sampling point x in layer l-1, Concat(∙) is the channel splicing operation; when l=L, When l=1, = .

[0102] In S6, the three-dimensional total variational regularized loss L TV Time consistency loss L time Calculate according to the following formulas respectively:

[0103] ,

[0104] In the formula, i, j, and k represent the spatial features V of the Lth layer, respectively. (L) The position index of a voxel in row, column, and height. , , , V (L) Voxel features at positions (i,j,k), (i+1,j,k), (i,j+1,k), and (i,j,k+1);

[0105] ,

[0106] In the formula, f(x,t+Δt) is the input vector of the sampling point at time t+Δt, MLP θ (∙) represents the predicted output of the MLP corresponding to the input vector, and Δt represents the time difference between the timestamps of adjacent image captures. It is the square of the L2 norm.

[0107] The training in S7 is specifically as follows:

[0108] A ray is randomly selected from dataset D, and coarse and fine sampling points are generated on the ray. For any sampling point, an input vector is generated according to step S5, and the vector is fed into the MLP to obtain the predicted volume density and predicted color. The total loss L is calculated, and the parameters of the multi-scale fusion NeRF network are adjusted by minimizing L.

[0109] The reconstruction of the new perspective image in S7 is as follows:

[0110] Generate rays from the camera origin to each pixel in the new viewpoint image;

[0111] Select any ray and generate sampling points on it. Pass the sampling points through the FPN-PAN multi-scale fusion unit and vector generation unit of the forestry 3D scene model to obtain the corresponding input vector. Then, pass the MLP to obtain the predicted volume density and predicted color, and calculate the color of the ray.

[0112] Each ray is processed sequentially to obtain its color, resulting in a new perspective image.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A NeRF modeling method based on multi-scale voxel fusion and total variational regularization, characterized in that, Includes the following steps; S1, construct dataset D; Use a camera to acquire N images of a 3D forestry scene from various perspectives, and then use the i-th image I... i Compared to the shooting timestamp t i Construct the i-th sample (I) i ,t i All samples constitute dataset D, I i From the camera origin o i The ray pointing to pixel p is r i,p The viewing direction is d i,p ; S2, performs coarse and fine sampling on each ray to generate N. c One coarse sampling point and N f Each fine sampling point; S3, using L voxels with progressively decreasing resolutions, divide the forestry 3D scene into L 3D meshes, extract voxel features from each mesh, and use all voxel features corresponding to the l-th resolution voxel as the l-th layer spatial feature V of the forestry 3D scene. (l) , 1≤l≤L; S4, for any sampling point x, generate its value in V. (l) Multiscale features are The sampling point x is either a coarse sampling point or a fine sampling point; S5. Construct a multi-scale fusion NeRF network, including FPN-PAN multi-scale fusion unit, vector generation unit and MLP; The FPN-PAN multi-scale fusion unit includes an FPN, a PAN, and a fusion layer; For sampling point x, features are extracted from the bottom up by FPN, and the FPN fusion feature of x at the l-th layer is generated. Then, features are extracted from the PAN from top to bottom, and the features from the (l-1)th layer of the PAN are fused in the l-th layer. and Aggregation yields the PAN fusion feature of x at layer l. The fusion layer is used to combine L elements. Channel-by-channel concatenation yields the global feature v of x. multi (x); The vector generation unit is used to generate the input vector at time t for each sampling point, where the input vector at time t for sampling point x is f(x,t). γ(x), γ(d), and γ(t) are the location of the sampling point x, the viewing direction of the ray, and the code corresponding to the shooting timestamp, respectively. The MLP is used as input to f(x,t) and outputs the predicted volume density and predicted color of x; S6, the total loss L of constructing a multi-scale fused NeRF network; ， In the formula, L recon For the MSE loss of true color and predicted color, L TV For three-dimensional total variational regularized loss, L time For time consistency loss, λ TV , λ time L respectively TV and L time The weights; S7. Using dataset D, a multi-scale fusion NeRF network is trained to minimize L, resulting in a forestry 3D scene model for reconstructing new perspective images in forestry 3D scenes. S2 specifically includes S21~S23: S21, in the ray r i,p Upper equal interval sampling N c The j-th coarse sample point is predicted by MLP from the given coarse sample points. volume density and color And calculate opacity and cumulative transmittance , 1≤j≤N c ; ， ， In the formula, exp(⋅) is the exp function. for With the next coarse sampling point The step size between, The opacity of the k-th coarse sampling point; S22, calculate the j-th coarse sampling point according to the following formula. sampling weights ; ， S23, will After normalization, the cumulative distribution function (CDF) is formed. Based on the CDF, N samples are uniformly sampled on [0,1]. f N random numbers are obtained by table lookup and interpolation. f Each fine sampling point.

2. The NeRF modeling method based on multi-scale voxel fusion and total variational regularization according to claim 1, characterized in that, In S3, a learnable 3D convolutional network is used to extract the voxel features of each 3D grid to obtain V. (l) .

3. The NeRF modeling method based on multi-scale voxel fusion and total variational regularization according to claim 1, characterized in that, In S4, the generated sampling point x is in V (l) Multiscale features in The method is as follows: In each layer V (l) The voxels to which the voxels belong are retrieved and multi-scale features are obtained by trilinear interpolation. ; ， In the formula, trilinear-interp(⋅) is the trilinear interpolation function.

4. The NeRF modeling method based on multi-scale voxel fusion and total variational regularization according to claim 1, characterized in that, In S5, the FPN is used to generate the FPN fusion feature of x at layer l according to the following formula. ; ， The PAN is used to generate the PAN fusion feature of x in the l-th layer according to the following formula. ; ， The fusion layer is used to generate global features of x according to the following formula. ; ， In the formula, UpConv(∙) is the upsampling convolution operation. The FPN fusion feature for sampling point x at layer l+1 is used, and DownConv(∙) is the downsampling convolution operation. For the PAN fusion feature of sampling point x in layer l-1, Concat(∙) is the channel splicing operation; when l=L, When l=1, = .

5. The NeRF modeling method based on multi-scale voxel fusion and total variational regularization according to claim 1, characterized in that, Three-dimensional total variational regularized loss L TV Time consistency loss L time Calculate according to the following formulas respectively: ， In the formula, i, j, and k represent the spatial features V of the Lth layer, respectively. (L) The position index of a voxel in row, column, and height. , , , V (L) Voxel features at positions (i,j,k), (i+1,j,k), (i,j+1,k), and (i,j,k+1); ， In the formula, f(x,t+Δt) is the input vector of the sampling point at time t+Δt, MLP θ (∙) represents the predicted output of the MLP corresponding to the input vector, and Δt represents the time difference between the timestamps of adjacent image captures. It is the square of the L2 norm.

6. The NeRF modeling method based on multi-scale voxel fusion and total variational regularization according to claim 1, characterized in that, The training in S7 is specifically as follows: A ray is randomly selected from dataset D, and coarse and fine sampling points are generated on the ray. For any sampling point, an input vector is generated according to step S5, and the vector is fed into the MLP to obtain the predicted volume density and predicted color. The total loss L is calculated, and the parameters of the multi-scale fusion NeRF network are adjusted by minimizing L.

7. The NeRF modeling method based on multi-scale voxel fusion and total variational regularization according to claim 1, characterized in that, The reconstruction of the new perspective image in S7 is as follows: Generate rays from the camera origin to each pixel in the new viewpoint image; Select any ray and generate sampling points on it. Pass the sampling points through the FPN-PAN multi-scale fusion unit and vector generation unit of the forestry 3D scene model to obtain the corresponding input vector. Then, pass the MLP to obtain the predicted volume density and predicted color, and calculate the color of the ray. Each ray is processed sequentially to obtain its color, resulting in a new perspective image.

Images