Incremental optimal view selection method for neural radiance field based on hybrid uncertainty estimation

Abstract

This invention discloses an incremental optimal view selection method for neural radiation fields based on mixed uncertainty estimation, comprising the following steps: inputting an RGB image set; calculating 3D position and viewing direction; initializing NeRF training; based on the 5D coordinates (x, d) of the remaining image set, threshold sampling is performed through a modified NeRF network to calculate the color and opacity of the RGB images, and then the color and opacity are integrated into a Beta distribution to calculate rendering uncertainty; inputting the 3D coordinates x of the remaining image set, a classifier is used to determine whether the trajectory is planar or non-planar, and the Voronoi diagram algorithm is used to estimate the position information to obtain the position uncertainty; selecting the optimal view, normalizing and summing the rendering uncertainty and position uncertainty to calculate the mixed uncertainty of each RGB image; selecting the image with the highest mixed uncertainty, adding it to the training set, and repeating this process until a specific reconstruction effect or a preset image quantity limit is achieved.

Description

Technical Field

[0001] This paper relates to the field of computer vision technology, specifically large-scale 3D reconstruction using Neural Radiance Fields (NeRF). More specifically, this invention relates to an incremental optimal view selection method based on hybrid uncertainty estimation using NeRF. Background Technology

[0002] Currently, image-based 3D reconstruction techniques are mainly divided into two categories: traditional geometry-based methods and neural network-based methods. Traditional methods primarily involve two steps: Structure From Motion (SfM) and Multiple View Stereo (MVS). SfM aims to analyze feature points from image sequences and obtain 3D structures and camera trajectories by adjusting parameters through bundle adjustment. Subsequently, MVS generates cloud points for a single viewpoint by estimating the depth map for each viewpoint. The other category, represented by Neural Radiance Fields (NeRF), represents a significant breakthrough in 3D reconstruction technology in recent years. NeRF implicitly expresses 3D scene information by training network parameters and can synthesize images from new perspectives. However, large-scale reconstruction technologies at the city level face two major challenges: First, large-scale reconstruction typically requires processing and storing massive amounts of data, which can quickly exhaust the memory resources of a single GPU, leading to slower processing speeds, insufficient memory, and other performance issues, making it particularly unfriendly to users or researchers with limited GPU memory resources; Second, with the increasing demand for real-time or near-real-time applications, such as navigation or disaster management, research into faster and more efficient large-scale 3D reconstruction technologies has become especially urgent.

[0003] To create more accurate, detailed, and useful large-scale 3D models and address the practical challenges of using large datasets and limited computing resources, this study proposes an efficient method for fast and robust large-scale 3D reconstruction with limited video memory resources. Summary of the Invention

[0004] Addressing the issues of lengthy training times and high memory requirements associated with existing large-scale reconstruction techniques, this invention provides a fast and efficient large-scale 3D reconstruction method based on neural radiation fields, capable of effectively handling large amounts of input data with limited memory resources. The proposed method mainly comprises two parts: an uncertainty-based view planning strategy and an information gain-based view selection strategy.

[0005] The method proposed in this invention fuses features of different time window sizes, uses multiple prediction heads to output multiple suggested positions at different levels, and takes the position of the point with the highest confidence as the final point tracking result. Moreover, the constructed model has no restrictions on the length of the input video sequence or the number of tracking points, and can complete the tracking of all points in all frames in parallel through only one forward propagation process.

[0006] To achieve the above objectives, this application adopts the following technical solution:

[0007] An incremental optimal view selection method for neural radiation fields based on mixed uncertainty estimation includes the following steps:

[0008] Steps for inputting an RGB image set: Input a set of N RGB image data taken by a drone at a uniform resolution;

[0009] Calculate 3D position and viewing direction; process each input RGB image, perform pose estimation, recover the structure in the RGB image, and output the 3D position x and viewing direction vector d corresponding to each image in the scene; the structure in the RGB image refers to the scene geometric information recovered from the RGB image, including but not limited to the position, shape and spatial relationship between objects;

[0010] The NeRF training initialization step involves randomly selecting a predetermined proportion of images from the RGB image set for initial training of the NeRF model; the predetermined proportion is no less than 15%.

[0011] The steps for calculating mixed uncertainty are as follows: Based on the 5D coordinates (x, d) of the remaining image set, threshold sampling is performed through a modified NeRF network to calculate the color and opacity of the RGB image. Then, the color and opacity are integrated into the Beta distribution to calculate the rendering uncertainty. The 3D coordinates x of the remaining image set are input, and a classifier is used to determine whether the trajectory is planar or non-planar. The Voronoi diagram algorithm is used to estimate the position information to obtain the position uncertainty.

[0012] The optimal view step is to normalize and sum the rendering uncertainty and position uncertainty to calculate the mixed uncertainty of each RGB image; the image with the highest mixed uncertainty is selected, added to the training set, and this process is repeated until a specific reconstruction effect or a preset image number limit is achieved.

[0013] The added training set is the image dataset used to train a NeRF model. NeRF, along with traditional triangular meshes, point clouds, and voxels, are all representations of 3D reconstruction. However, while triangular meshes, point clouds, and voxels are explicit representations, NeRF uses implicit representations, storing scene information within a neural network. To reconstruct a scene, a dedicated network needs to be trained, using acquired images of that scene for multi-view reconstruction. The images used for reconstruction constitute the training set. With sufficient GPU memory, the more images in the training set, the more information they provide, and the better the reconstruction results.

[0014] As a preferred aspect of the present invention, in the step of calculating mixed uncertainties...

[0015] Based on the 5D coordinates (x, y, z, θ, φ) of the remaining image set, threshold sampling is performed using a modified NeRF network, and multi-resolution hash information is stored using an explicit voxel grid and a hidden neural network. The color and opacity of the sampled points on each ray of each RGB image are calculated, and the rendering uncertainty of each image is calculated by integrating these values ​​into a Beta distribution. Simultaneously,

[0016] Input the 3D coordinates (x, y, z) of the remaining image set, and use a classifier to determine whether the UAV flight trajectory is planar or non-planar. For planar trajectories, use Voronoi information radiation field to perform top-level global planning and bottom-level local planning to compress and limit the position information. For non-planar trajectories, use Voronoi clustering algorithm to quantify the uncertainty of position by the distance from the center point.

[0017] Here, (x,d) and (x,y,z,θ,φ) are different descriptions of 5D coordinates. In the former, x and d represent the position and viewing direction in three-dimensional space, respectively. In the latter, x,y,z refer to the values ​​of the xyz coordinate system in three-dimensional space, and θ,φ refer to the horizontal and vertical components of the direction.

[0018] As a preferred aspect of the present invention, the step of calculating mixed uncertainty is specifically performed as follows:

[0019] Using the NeRF model, the color and opacity of each ray are obtained, and the volume density is calculated. The formula for calculating the volume density is as follows:

[0020]

[0021] Among them, a i σ represents the volume density at the i-th sampling point on the ray. j δ represents the opacity of the j-th sampling point. j This represents the distance between the j-th sampling point and the previous sampling point.

[0022] The color variance is calculated for each ray using the NeRF model; the formula for calculating the color variance is as follows:

[0023] β 2 (r(t i ))=-P(α i )log P(α i )

[0024] Where, β 2 (r(t i )) represents the variance of the i-th sampling point on the ray, α i P(α) represents the volume density of the i-th sampling point (Formula 1 above). i ) represents α i The total volume density of this ray ∑α i The proportion;

[0025] Construct a Beta distribution based on the estimated mean and variance. in, The color representing a predicted ray. The average color of a ray. This represents the mean of the variance of a ray;

[0026] The continuous function is optimized by minimizing the squared reconstruction error between the ground truth RGB image and the rendered pixel colors, while simultaneously calculating the rendering uncertainty for each image; the formula for calculating image rendering uncertainty is as follows:

[0027]

[0028] N represents the rendering uncertainty of the i-th image. r This represents the total number of rays in each image. This represents the squared error between the true RGB image and the rendered pixel color. The average variance of ray i;

[0029] Calculate the Hausdorff distance between two datasets to determine whether the image trajectory is planar or non-planar; where the Hausdorff distance is... h(A,B) is the Hasdorf distance between two trajectories A and B, ||a i -b j || represents the distance between the i-th point in trajectory A and the j-th point in trajectory B.

[0030] As a preferred aspect of the present invention, the step of calculating the mixed uncertainty is specifically performed as follows: for a planar trajectory, the position uncertainty is calculated using the following formula:

[0031]

[0032] Among them, F p (I) represents the planar position uncertainty value of photograph I, N v This represents the total number of 3D pose points. λ represents the weight values ​​of points i and j at point i. i The distance value below, A i Let i represent the area of ​​the Venn diagram at point i.

[0033] As a preferred aspect of the present invention, for non-planar trajectories, the following formula is used to estimate the position uncertainty:

[0034]

[0035] Among them, F np (I) represents the nonplanar positional uncertainty value of photograph I, G i It is the probability of selecting image i based on the Venn diagram polygon, r o This represents the relative local density of the evaluation point before and after selection.

[0036] In a preferred aspect of the present invention, in the step of selecting the optimal view, rendering uncertainty and position uncertainty are normalized and summed to form a mixed uncertainty for each image; by selecting the image with the highest mixed uncertainty from the candidate set and adding it to the training set, and repeating this process until a specific reconstruction effect or a preset image quantity limit is achieved, the view selection strategy is optimized; wherein, the formula for calculating the mixed uncertainty is as follows:

[0037]

[0038] in, This represents the positional uncertainty of image I. The rendering uncertainty of image I is represented by the sum of the two normalized values, which is the total uncertainty ψ of image I. 2 (I).

[0039] As a preferred aspect of the present invention, it further includes:

[0040] Iterative optimization steps: In each iteration, the uncertainty of each unused candidate view is calculated based on the current training set, the view with the largest information gain is selected and added to the training set to gradually improve the quality of new view synthesis.

[0041] As a preferred aspect of the present invention, in the step of inputting the RGB image set, a rotary-wing UAV is first used to fly a specific route over the target area, and the flight route is divided into planar and non-planar routes; during the flight, fixed-point shooting is performed to obtain a set of N aerial survey RGB image data of the same resolution size, taken by the UAV, whether orthogonal or oblique.

[0042] As a preferred aspect of the present invention, in the calculation of 3D position and viewing direction, COLMAP software is used to perform pose estimation on all input RGB images to obtain the camera shooting pose corresponding to each image; wherein, the camera model is set to a pinhole camera model by default and all cameras are set to use the same intrinsic parameters for incremental SfM reconstruction, thereby obtaining the intrinsic and extrinsic parameters of all cameras.

[0043] As a preferred aspect of the present invention, in the initialization NeRF training step, at least 15% of the images are randomly selected from the RGB image set for initialization training of the NeRF model, so that the NeRF model learns the basic structure and appearance features of the scene and stores them in the multilayer perceptron network.

[0044] In this process, 15% of the images in the training set are randomly selected for initial training to obtain a problematic scene model. Subsequent training is then performed iteratively based on this initial model. If the problem is minor, fewer iterations are needed to achieve good results; otherwise, more iterations are required.

[0045] For example: If there are 100 images, select 10 (10%) images as the test set to evaluate the model's performance, and use the remaining 90 images as the training set. Of these, 15 (15%) images are used for initialization training, and the rest are training candidate images. After initialization training, a poor scene model is obtained. Then, calculate the mixture uncertainty of all images in the training candidate set to obtain the candidate image with the most information, and add it to the training set (the training set now consists of 16 images: 15 initial images + 1 newly added candidate image). Iterative training is then performed to obtain a new optimized scene model. The above steps are repeated until the set conditions are met, such as selecting 15 images.

[0046] The specific advantages of this invention are as follows:

[0047] This invention introduces a method for achieving information gain based on a hybrid uncertainty of positional and rendering uncertainties. Furthermore, by selecting the viewpoint that maximizes gain, it facilitates rapid, large-scale 3D reconstruction with limited video memory resources. Compared to existing technologies, it offers significant advantages, including the ability to handle larger input data volumes, faster large-scale 3D reconstruction speed, and higher rendered image quality under the same conditions.

[0048] This incremental optimal view selection method for neural radiation fields based on hybrid uncertainty estimation can solve the artifact problem in multi-view scenarios and reduce computational costs while maintaining high rendering realism. This method specifically focuses on how to effectively select the view that brings the maximum information gain from candidate views under the constraint of limited video memory resources, thereby improving rendering quality and efficiency.

[0049] Furthermore, this invention combines hybrid multi-resolution hash storage of explicit voxel meshes and implicit neural networks, Voronoi diagram information gain radiation field and clustering algorithms, threshold sampling, and flight classifiers to further improve processing efficiency while ensuring the performance of the original NeRF network. This technology can be used as a plug-in tool to assist existing systems in achieving more efficient 3D scene reconstruction and rendering.

[0050] Specific embodiments of the invention are disclosed in detail with reference to the following description and accompanying drawings, indicating how the principles of the invention can be employed. It should be understood that the embodiments of the invention are not therefore limited in scope. Within the spirit and scope of the appended claims, embodiments of the invention include many changes, modifications, and equivalents.

[0051] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments.

[0052] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, whole, step, or component, but does not exclude the presence or addition of one or more other features, wholes, steps, or components. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1This is a schematic diagram of the incremental optimal view selection method for neural radiation fields based on mixed uncertainty estimation, as proposed in this invention.

[0055] Figure 2 This is a flowchart of the incremental optimal view selection method for neural radiation fields based on mixed uncertainty estimation, as described in this invention.

[0056] Figure 3 This is a framework diagram of the construction method of the incremental optimal view selection method for neural radiation fields based on mixed uncertainty estimation of the present invention.

[0057] Figure 4 This is a schematic diagram of the Veno diagram information gain radiation field used in the incremental optimal view selection method for neural radiation field based on mixed uncertainty estimation in this invention.

[0058] Figure 5 This is a schematic diagram of the Veno diagram clustering algorithm used in the incremental optimal view selection method for neural radiation fields based on mixed uncertainty estimation in this invention.

[0059] Figure 6 This is a demonstration of the effectiveness of some results in the incremental optimal view selection method for neural radiation fields based on mixed uncertainty estimation of the present invention. Detailed Implementation

[0060] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0061] like Figures 1 to 6 As shown in the figure, one embodiment of the present invention proposes an incremental optimal view selection method for neural radiation fields based on hybrid uncertainty estimation, mainly used to solve the problems of large-scale 3D reconstruction and new perspective rendering output. The algorithm is illustrated in the figure. Figure 1 As shown.

[0062] The algorithm of this invention mainly consists of two parts: calculating rendering uncertainty and calculating position uncertainty and obtaining mixed uncertainty to perform viewpoint selection planning with maximum information gain. The algorithm flow is as follows: Figure 2 As shown, the main steps include:

[0063] S1. First, input a set of real RGB photos. Input a set of high-resolution RGB image data taken using a drone.

[0064] Rotary-wing drones are used to fly along specific routes over a given area. These routes can be either planar or non-planar; non-planar routes are typically flown manually. Planar routes can be set in an app, usually using a tic-tac-toe pattern. During the flight, fixed-point images are taken, resulting in a set of N aerial survey images of the same resolution, captured by the drone, whether orthogonal or oblique.

[0065] S2. Use COLMAP to perform pose estimation (camera position and orientation in space) and sparse point cloud reconstruction on all input images to obtain the camera shooting pose and sparse point cloud information corresponding to each image.

[0066] Using COLMAP to estimate the pose (camera position and orientation in space) of all input images yields the camera pose for each image. The default camera model is a pinhole camera model (PINEHOLE), and all cameras are configured to use the same intrinsic parameters for incremental SfM reconstruction, thus obtaining the intrinsic and extrinsic parameters of all cameras.

[0067] S3. Initialize NeRF training. Randomly select a certain proportion of images from the input dataset (at least 15% is recommended) for initial training of the NeRF model.

[0068] At least 15% of the images in the input dataset are randomly selected for initial training of the NeRF model to ensure that the model can learn the basic structure and appearance features of the scene.

[0069] S4. Calculate the mixed uncertainty. Based on the 5D coordinates (x,d) of the remaining image set, threshold sampling is performed using a modified NeRF network to calculate the color and opacity of the image.

[0070] The NeRF network is the part of the NeRF model responsible for learning and encoding scene information. The NeRF model relies on the NeRF network to learn the 3D representation of the scene, and other components (such as volume rendering equations) are also needed to achieve new perspective synthesis and 3D reconstruction.

[0071] The complete image set is divided into a test set, a training set, and a training candidate set. The remaining image set refers to the training candidate set. The test set does not participate in the training process and is only used to evaluate the training results. For example, if there are 100 images, 10 images (10%) are selected as the test set to evaluate the model's performance, and the remaining 90 images are used as the training candidate set. First, 15 images (15%) are selected for initial training, and then the images with the most information are selected from the remaining training candidate set and added to the training set.

[0072] Then, color and opacity are integrated into the Beta distribution to calculate rendering uncertainty. For positional uncertainty, the 3D coordinates x of the remaining image set are input, a classifier determines whether the trajectory is planar or non-planar, and then the Voronoi diagram algorithm is used to estimate the positional information to obtain the positional uncertainty. The specific algorithm for this step is as follows:

[0073] S41. Using the NeRF model's MLP, calculate the color and opacity of each ray. The subsequent volume density calculation takes into account the distance between adjacent samples and the opacity. The volume density calculation formula is as follows:

[0074]

[0075] Among them, a i σ represents the volume density at the i-th sampling point on the ray. j δ represents the opacity of the j-th sampling point. j This represents the distance between the j-th sampling point and the previous sampling point.

[0076] S42. Using the NeRF model, calculate the color variance for each ray. The formula for calculating color variance is as follows:

[0077] β 2 (r(t i ))=-P(α i )log P(α i )

[0078] Where, β 2 (r(t i )) represents the variance of the i-th sampling point on the ray, α i P(α) represents the volume density of the i-th sampling point (Formula 1 above). i ) represents α i The total volume density of this ray ∑α i The proportion.

[0079] S43. Since the volume density at a specific location is only affected by its own 3D coordinates and not by the viewing direction, the distribution at different locations is independent of each other. At the same time, volume rendering can be approximated as a linear combination of light sampling points, so a Beta distribution can be constructed based on the estimated mean and variance.

[0080] in, The color representing the predicted (calculated according to the constructed model) ray. The average color of a ray. This represents the mean of the variance of a ray, and a beta distribution is constructed.

[0081] S44. Optimize the continuous function by minimizing the squared reconstruction error between the true RGB image and the rendered pixel colors, while simultaneously calculating the rendering uncertainty for each image. The formula for calculating image rendering uncertainty is as follows:

[0082]

[0083] N represents the rendering uncertainty of the i-th image. r This represents the total number of rays in each image. This represents the squared error between the true RGB image and the rendered pixel color. The average variance of ray i is represented.

[0084] S45. Calculate the Hausdorff distance between two datasets. To determine whether the image trajectory is planar or non-planar.

[0085] Where h(A,B) is the Hasdorf distance between the two trajectories A and B, ||a i -b j || represents the distance between the i-th point in trajectory A and the j-th point in trajectory B.

[0086] S46. For planar trajectories, the location uncertainty is calculated using a Voronoi diagram, taking into account the importance of each location point and the area of ​​the planar Voronoi diagram. The location uncertainty is calculated using the following formula:

[0087]

[0088] Among them, F p (I) represents the planar position uncertainty value of photograph I, N v This represents the total number of 3D pose points. λ represents the weight values ​​of points i and j at point i. i The distance value below, A i Let i represent the area of ​​the Venn diagram at point i.

[0089] For non-planar trajectories, the Voronoi diagram clustering algorithm is used to estimate positional uncertainty, taking into account both three-dimensional spatial importance and local correlation density. In this case, positional uncertainty is calculated using the following formula:

[0090]

[0091] Among them, F np (I) represents the nonplanar positional uncertainty value of photograph I, G i It is the probability of selecting image i based on the Venn diagram polygon, r iThis represents the relative local density of the evaluation point before and after selection.

[0092] S5. Select the optimal view. Normalize and sum the rendering uncertainty and location uncertainty to calculate the mixed uncertainty for each image. Select the image with the highest mixed uncertainty, add it to the training set, and repeat this process until a specific reconstruction effect or a preset image quantity limit is achieved. The formula for calculating the mixed uncertainty is as follows:

[0093]

[0094] in, This represents the positional uncertainty of image I. The rendering uncertainty of image I is represented by the sum of the two normalized values, which is the total uncertainty ψ of image I. 2 (I).

[0095] S6. Iterative Optimization. In each iteration, the uncertainty of each unused candidate view is calculated based on the current training set. The view with the largest information gain is selected and added to the training set to gradually improve the quality of the new perspective synthesis.

[0096] It's worth noting that NeRF introduces a novel method for scene representation and view synthesis, allowing the synthesis of highly realistic 3D scenes from 2D images. The training process involves capturing a set of 2D images of the scene from different viewpoints. For each pixel in the image, NeRF computes the corresponding ray in 3D space. NeRF estimates the scene color and opacity at each point along these rays, then compares the predicted appearance color values ​​with the observed appearance color values ​​and adjusts the network parameters to minimize the differences. After training, NeRF is able to synthesize new scene views by projecting rays from a virtual camera position and accumulating color and opacity along the rays using the learned neural radiation field. This process allows for the generation of photographs from previously unseen viewpoints, providing realistic scene rendering images. Instant-NGP, proposed by NVIDIA, is a highly engineering-valued and currently the fastest neural radiation field model. This work proposes a multi-resolution hashing encoding to accelerate model training, and the multi-resolution hash table augmentation of trainable feature vectors further reduces the model size. Furthermore, the entire system is implemented using a fully fused CUDA kernel, maximizing parallelism.

[0097] In this embodiment, each iteration of training is based on a 3D reconstruction method using neural radiation fields, which takes into account both positional and rendering uncertainties in viewpoint planning. The specific model used is Instant-NGP.

[0098] S6. Iterative Optimization. In each iteration, the uncertainty of each unused candidate view is calculated based on the current training set. The view with the largest information gain is selected and added to the training set to gradually improve the quality of the new perspective synthesis.

[0099] Depend on Figure 6 As can be seen, some results of the fast large-scale 3D reconstruction method based on neural radiation field of the present invention selected six sets of publicly available large scene datasets and compared them with the previous baseline, all of which achieved better reconstruction results.

[0100] In this application, multiple elements, components, parts, or steps can be provided by a single integrated element, component, part, or step. Alternatively, a single integrated element, component, part, or step can be divided into multiple separate elements, components, parts, or steps. The use of "a" or "an" to describe an element, component, part, or step does not imply the exclusion of other elements, components, parts, or steps.

[0101] It should be understood that the above description is for illustrative purposes and not for limitation. Many embodiments and applications beyond the provided examples will be apparent to those skilled in the art upon reading the above description. Therefore, the scope of this teaching should not be determined by reference to the above description, but rather by reference to the appended claims and the full scope of their equivalents. For purposes of completeness, all articles and references, including patent applications and publications, are incorporated herein by reference. The omission of any aspect of the subject matter disclosed herein in the preceding claims is not intended as a waiver of that subject matter, nor should it be construed as an indication that the inventors have not considered that subject matter as part of the disclosed inventive subject matter.

Claims

1. A method for incremental optimal view selection of neural radiation fields based on mixed uncertainty estimation, characterized in that, Includes the following steps: Steps for inputting an RGB image set: Input a set of N RGB image data taken by a drone at a uniform resolution; Calculate 3D position and viewing direction; Each input RGB image is processed to estimate its pose, recover the structure in the RGB image, and output the 3D position x and the viewing direction vector d corresponding to each image in the scene. The NeRF training initialization step involves randomly selecting a predetermined proportion of images from the RGB image set for initial training of the NeRF model; the predetermined proportion is no less than 15%. The steps for calculating mixed uncertainty are as follows: Based on the 5D coordinates (x, d) of the remaining image set, threshold sampling is performed through a modified NeRF network to calculate the color and opacity of the RGB image. Then, the color and opacity are integrated into the Beta distribution to calculate the rendering uncertainty. The 3D coordinates x of the remaining image set are input, and a classifier is used to determine whether the trajectory is planar or non-planar. The Voronoi diagram algorithm is used to estimate the position information to obtain the position uncertainty. The optimal view step is to normalize and sum the rendering uncertainty and position uncertainty to calculate the mixture uncertainty of each RGB image; the image with the highest mixture uncertainty is selected, added to the training set, and this process is repeated until a specific reconstruction effect or a preset image number limit is achieved. In the step of calculating mixed uncertainties, Based on the 5D coordinates (x, y, z, θ, φ) of the remaining image set, threshold sampling is performed using a modified NeRF network, and multi-resolution hash information is stored using an explicit voxel grid and a hidden neural network. The color and opacity of the sampled points on each ray of each RGB image are calculated, and the rendering uncertainty of each image is calculated by integrating these values ​​into a Beta distribution. Simultaneously, Input the 3D coordinates (x, y, z) of the remaining image set, and use a classifier to determine whether the UAV flight trajectory is planar or non-planar. For planar trajectories, use Voronoi information radiation field to perform top-level global planning and bottom-level local planning to compress and limit the position information. For non-planar trajectories, use Voronoi clustering algorithm to quantify the uncertainty of position by the distance from the center point. The step of calculating mixed uncertainty is specifically performed as follows: Using the NeRF model, the color and opacity of each ray are obtained, and the volume density is calculated. The formula for calculating the volume density is as follows: ； in, This represents the volume density at the i-th sampling point on the ray. The opacity of the j-th sampling point is represented by This represents the distance between the j-th sampling point and the previous sampling point; Represents the opacity of the i-th sampling point. This represents the distance between the i-th sampling point and the previous sampling point; The color variance is calculated for each ray using the NeRF model; the formula for calculating the color variance is as follows: ； in, This represents the variance of the i-th sampling point on the ray. Represents the volume density of the i-th sampling point. represent The total volume density of this ray The proportion; Construct a Beta distribution based on the estimated mean and variance. ;in, The color representing a predicted ray. The average color of a ray. This represents the mean of the variance of a ray; The continuous function is optimized by minimizing the squared reconstruction error between the ground truth RGB image and the rendered pixel colors, while simultaneously calculating the rendering uncertainty for each image; the formula for calculating image rendering uncertainty is as follows: ； This represents the rendering uncertainty of the i-th image. This represents the total number of rays in each image. This represents the squared error between the true RGB image and the rendered pixel color. The average variance of ray i; Calculate the Hausdorff distance between two datasets to determine whether the image trajectory is planar or non-planar; where the Hausdorff distance is... ; Let the Hasdorf distance be the distance between two trajectories A and B. Let be the distance between the i-th point in trajectory A and the j-th point in trajectory B.

2. The incremental optimal view selection method for neural radiation fields based on mixed uncertainty estimation as described in claim 1, characterized in that, The step of calculating the mixed uncertainty is specifically performed as follows: For a planar trajectory, the position uncertainty is calculated using the following formula: ； in, Let I be the planar position uncertainty value of photo I. This represents the total number of 3D pose points. This represents the weight values ​​of points i and j at point i. The distance value below, Let i represent the area of ​​the Venn diagram at point i.

3. The incremental optimal view selection method for neural radiation fields based on mixed uncertainty estimation as described in claim 1, characterized in that, For non-planar trajectories, the following formula is used to estimate position uncertainty: ； in, Let I be the nonplanar positional uncertainty value of photo I. It is the probability of selecting image i based on the polygon of the Venn diagram. This represents the relative local density of the evaluation point before and after selection.

4. The incremental optimal view selection method for neural radiation fields based on hybrid uncertainty estimation as described in claim 1, characterized in that, In the optimal view selection step, rendering uncertainty and location uncertainty are normalized and summed to form the mixed uncertainty of each image. The view selection strategy is optimized by selecting the image with the highest mixed uncertainty from the candidate set, adding it to the training set, and repeating this process until a specific reconstruction effect or a preset image quantity limit is achieved. The formula for calculating the mixed uncertainty is as follows: ； in, This represents the positional uncertainty of image I. This represents the rendering uncertainty of image I. The sum of the two after normalization is the total uncertainty of image I. .

5. The incremental optimal view selection method for neural radiation fields based on hybrid uncertainty estimation as described in claim 1, characterized in that, Also includes: Iterative optimization steps; In each iteration, the uncertainty of each unused candidate view is calculated based on the current training set, the view with the largest information gain is selected and added to the training set to gradually improve the quality of new perspective synthesis.

6. The incremental optimal view selection method for neural radiation fields based on mixed uncertainty estimation as described in claim 1, characterized in that, In the step of inputting the RGB image set, a rotary-wing UAV is first used to fly over the target area along a specific flight path, which is divided into planar and non-planar paths. During the flight, fixed-point shooting is performed to obtain a set of N aerial survey RGB image data of the same resolution size, taken by the UAV, whether orthogonal or oblique.

7. The incremental optimal view selection method for neural radiation fields based on mixed uncertainty estimation as described in claim 1, characterized in that, In the calculation of 3D position and viewing direction, COLMAP software is used to estimate the pose of all input RGB images to obtain the camera shooting pose corresponding to each image; wherein, the camera model is set to a pinhole camera model by default and all cameras are set to use the same intrinsic parameters for incremental SfM reconstruction, thereby obtaining the intrinsic and extrinsic parameters of all cameras.

8. The incremental optimal view selection method for neural radiation fields based on hybrid uncertainty estimation as described in claim 1, characterized in that, In the initialization NeRF training step, at least 15% of the images are randomly selected from the RGB image set for initial training of the NeRF model, so that the NeRF model learns the basic structure and appearance features of the scene and stores them in the multilayer perceptron network.

Images