Method for acquiring photos by using sparse density unmanned aerial vehicle to improve precision of erf algorithm

Abstract

The invention relates to a method for acquiring photos by using a closeness-density unmanned aerial vehicle to improve the precision of a nerf algorithm, and belongs to the technical field of three-dimensional reconstruction and neural rendering. The method comprises the following steps: collecting multi-view photos in a target scene in a sparse density path through an unmanned aerial vehicle; preprocessing the collected photos, extracting image features and generating sparse point clouds; inputting the extracted features and point cloud data to call a sparse image set to train a NeRF model, and generating high-quality three-dimensional scene reconstruction; by analyzing the deviation between a reconstruction result and a real scene, an unmanned aerial vehicle acquisition path and camera parameters are optimized; according to the optimized path, photos are collected again, the NeRF model is iteratively updated, unmanned aerial vehicle photo collection optimization based on the sparse image set is achieved, the reconstruction precision of the NeRF algorithm is improved, and the method is suitable for a high-precision three-dimensional modeling scene.

Description

Technical Field

[0001] This invention belongs to the field of 3D reconstruction and neural rendering technology, specifically involving a method for improving the accuracy of the NERF algorithm by using sparse-density UAVs to collect photos. Background Technology

[0002] Neural Radiation Field (NeRF) algorithms generate high-quality 3D scene reconstructions from multi-view images and have been widely applied in virtual reality, augmented reality, and robot navigation. However, traditional NeRF algorithms are highly dependent on the quantity and quality of input images, typically requiring dense image acquisition to ensure reconstruction accuracy. This leads to high acquisition costs, computational complexity, and low efficiency in practical applications. Drones, as flexible image acquisition tools, can acquire multi-view data in complex scenes. However, existing drone acquisition methods often employ fixed paths or dense sampling, making it difficult to adapt to dynamic scene changes, resulting in low acquisition efficiency or insufficient reconstruction quality. Sparse-density acquisition techniques offer new possibilities for optimizing drone acquisition paths and improving the accuracy of the NeRF algorithm. By analyzing scene features and reconstruction errors, the NeRF model can predict the optimal acquisition strategy, thereby achieving efficient 3D modeling. This invention utilizes these techniques to propose a method for improving the accuracy of the NeRF algorithm by using sparse-density drone image acquisition, thereby enhancing the reconstruction accuracy and acquisition efficiency of the NeRF algorithm. Summary of the Invention

[0003] To address the aforementioned problems in the existing technology, this invention provides a method for improving the accuracy of the NERF algorithm by using densely packed drones to acquire images. The objective of this invention can be achieved through the following technical solutions: S1: Based on the image acquisition module on the drone, multi-view images are acquired in the target scene using a preset density path, and an initial sparse point cloud is constructed through synchronous positioning technology; S2: Based on the multi-view images and sparse point clouds, call the sparse image set to train the neural radiation field model, generate the three-dimensional scene reconstruction result according to the multi-view images, and evaluate the reconstruction accuracy; S3: Based on the reconstruction accuracy assessment results, combined with the UAV flight status and scene complexity, the density is adjusted by planning the UAV flight path using the particle swarm algorithm to generate the best acquisition scheme for the current scene. S4: Analyze the optimal acquisition scheme through the quality reconstruction evaluation module, and perform detection based on evaluation indicators such as geometric consistency, texture detail, and depth error; if the reconstruction accuracy is found to be below expectations, automatically adjust the density and neural radiation field model parameters of the UAV.

[0004] Specifically, the image acquisition module includes a high-resolution camera, a synchronous positioning module, and an attitude calibration unit. The high-resolution camera is used to capture multi-view images of the target scene. The synchronous positioning module constructs a scene point cloud through synchronous positioning technology. The attitude calibration unit corrects the camera shooting angle through inertial measurement data.

[0005] Specifically, the sparse point cloud is generated based on a multi-view stereo reconstruction algorithm. By fusing depth information between multi-view images, the spatial structure is restored, providing initial geometric constraints for the neural radiation field model and reducing the convergence time in the early stages of training.

[0006] Specifically, the particle swarm optimization algorithm uses scene complexity and reconstruction accuracy as objective functions, and adjusts the drone's flight altitude and shooting angle by analyzing the geometric and texture complexity of the scene.

[0007] Specifically, the quality reconstruction evaluation module includes a geometric consistency evaluation unit, a texture detail analyzer, and a depth error calculator. The geometric consistency evaluation unit calculates the geometric error by comparing the point cloud deviation between the reconstructed model and the real scene. The texture detail analyzer evaluates the texture reconstruction quality by analyzing the image pixel distribution. The depth error calculator quantifies the depth prediction deviation by comparing the depth map.

[0008] Specifically, the neural radiation field model is trained using a sparse image set. It takes pixel features from multi-view photographs and sparse point clouds as input, and bidirectionally models to obtain the spatial context features of the scene. This is used to reconstruct the 3D geometry and texture details of the captured scene. The formula for calculating the color of the neural radiation field volume rendering is as follows: , Where C^(r) is the predicted pixel color corresponding to the ray, N is the number of sampling points on the ray, σi is the volume density of the i-th sampling point, δi is the distance between the i-th point and the (i+1)-th point, and ci is the color of the i-th point in the viewing direction. The training loss function for neural radiation fields is: , Where Lcolor is the color regression loss, C^(r) is the model predicted color, Cgt(r) is the real image pixel color, and R is the set of rays sampled in the current training batch.

[0009] Specifically, the synchronous positioning technology analyzes the geometric relationships between the multi-view images, uses triangulation to calculate the three-dimensional spatial position of feature points, and constructs an initial sparse point cloud based on the reprojection error.

[0010] Specifically, the density-sparse path is dynamically balanced between coverage, energy consumption and reconstruction accuracy through a multi-objective optimization strategy based on the UAV's current flight attitude, remaining battery power, wind speed disturbance and flight safety boundary, to generate a set of density-sparse acquisition paths that meet the task constraints.

[0011] Specifically, the image acquisition module also includes an image quality assessment unit, which is used to automatically filter out overexposed, blurry, and highly occluded images after image acquisition, and to remove low-quality images before neural radiation field training.

[0012] Specifically, the neural radiation field model training process introduces a region sampling weight mechanism, which assigns a higher training frequency to image regions with high acquisition density.

[0013] The beneficial effects of this invention are as follows: This invention proposes a UAV image acquisition method combining sparse and dense image acquisition strategies, significantly improving the accuracy, efficiency, and stability of 3D reconstruction based on the Neural Radiation Field (NeRF) algorithm. By pre-setting sparse-density flight paths within the scene, the UAV can achieve a combination of sparse acquisition across the entire area and dense acquisition of key regions, balancing broad coverage with local detail representation. During the acquisition phase, an initial sparse point cloud is constructed by combining simultaneous localization technology and image pose information, providing accurate spatial structure constraints for the subsequent reconstruction process, reducing the convergence time of the NeRF model in the early stages of training, and improving the stability of 3D modeling. During neural network training, the introduction of sparse image sets as prior information not only accelerates the model's overall perception of scene geometry but also enhances the model's texture restoration capability in high-density acquisition areas through a region sampling weighting mechanism, thereby effectively alleviating the blurring problem of traditional NeRF methods in areas with complex textures or abrupt structural changes. The introduction of particle swarm optimization (PSO) makes UAV flight path planning more intelligent and adaptive, dynamically adjusting the density of image capture based on scene complexity, reconstruction accuracy feedback, and equipment status. This enables refined data acquisition and resource allocation. Furthermore, the feedback analysis module designed in this invention supports automatic evaluation of reconstruction errors after model training, including indicators such as geometric consistency, texture detail, and depth deviation. If the reconstruction effect is found to be unsatisfactory, a closed-loop mechanism can be used to re-optimize the flight strategy and model parameters, forming an automated and intelligent accuracy improvement system. In summary, the method of this invention has advantages such as strong image acquisition intelligence, high modeling accuracy, high resource utilization, and good system robustness. It is suitable for applications requiring high 3D modeling accuracy, such as urban modeling, cultural relic digitization, post-disaster reconstruction, and unmanned surveying. Attached Figure Description

[0014] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0015] Figure 1 This is a flowchart illustrating a method for improving the accuracy of the NERF algorithm using sparse-density UAVs to acquire photos, according to the present invention. Detailed Implementation

[0016] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0017] Please see Figure 1 A method to improve the accuracy of the NERF algorithm by using densely packed drones to collect images: S1: Based on the image acquisition module on the drone, multi-view images are acquired in the target scene using a preset density path, and an initial sparse point cloud is constructed through synchronous positioning technology; S2: Based on the multi-view images and sparse point clouds, call the sparse image set to train the neural radiation field model, generate the three-dimensional scene reconstruction result according to the multi-view images, and evaluate the reconstruction accuracy; S3: Based on the reconstruction accuracy assessment results, combined with the UAV flight status and scene complexity, the density is adjusted by planning the UAV flight path using the particle swarm algorithm to generate the best acquisition scheme for the current scene. S4: Analyze the optimal acquisition scheme through the quality reconstruction evaluation module, and perform detection based on evaluation indicators such as geometric consistency, texture detail, and depth error; if the reconstruction accuracy is found to be below expectations, automatically adjust the density and neural radiation field model parameters of the UAV.

[0018] In the real-time 3D reconstruction embodiment in the closed environment of an industrial park, during the deployment phase, the system first divides the park into two types of areas based on the park's CAD map, facility layout diagram, and historical monitoring point data, through structural complexity analysis and maintenance priority assessment: dense inspection area: including areas with dense machinery operation, open-air equipment, and high-temperature operation areas, which require high-frequency and detailed modeling; sparse coverage area: such as roads, parking lots, and vacant areas, which only require low-density modeling and coverage. A swarm of drones equipped with IMU+GPS positioning modules, high dynamic range imaging (HDR) cameras, and obstacle avoidance radar takes off and performs data collection tasks according to preset density paths. The control system assigns local area tasks to each UAV. Dense areas use a 5-meter spacing grid path with multi-angle overhead / side-view shooting nodes; sparse areas use a 15-meter spacing regular grid path for rapid traversal scanning. All image data undergoes synchronous positioning processing and pre-screening through the edge computing module, automatically removing blurry, highly reflective, and occluded image frames, retaining high-quality images to construct a sparse image set. Based on synchronous positioning technology and image feature matching, the structure from motion (SfM) algorithm is used to recover the camera pose of each frame of UAV images, and combined with GPS trajectory to construct an initial sparse point cloud as a spatial prior for the neural radiation field model. To further improve modeling efficiency, this method designs a region weighting mechanism during the training process: the sampling frequency of dense area images is adjusted to 3 times that of sparse areas, and a learnable attention weight mechanism is used to dynamically focus on high-error areas during the image feature fusion stage. Meanwhile, a depth consistency loss term is introduced to enhance the geometric constraints of depth prediction, enabling NeRF training to achieve both pixel-level texture restoration and spatial structure fidelity. After the model training is completed, the error evaluation module is used to evaluate the 3D model. If the texture of a certain block is found to be incomplete or the structure reconstruction error (RMSE) exceeds the set threshold (such as 10cm), the particle swarm optimization (PSO) algorithm in the system is called to perform local replanning of the flight path.

[0019] Specifically, the image acquisition module includes a high-resolution camera, a synchronous positioning module, and an attitude calibration unit. The high-resolution camera is used to capture multi-view images of the target scene. The synchronous positioning module constructs a scene point cloud through synchronous positioning technology. The attitude calibration unit corrects the camera shooting angle through inertial measurement data.

[0020] Specifically, the sparse point cloud is generated based on a multi-view stereo reconstruction algorithm. By fusing depth information between multi-view images, the spatial structure is restored, providing initial geometric constraints for the neural radiation field model and reducing the convergence time in the early stages of training.

[0021] Specifically, the particle swarm optimization algorithm uses scene complexity and reconstruction accuracy as objective functions, and adjusts the drone's flight altitude and shooting angle by analyzing the geometric and texture complexity of the scene.

[0022] Specifically, the quality reconstruction evaluation module includes a geometric consistency evaluation unit, a texture detail analyzer, and a depth error calculator. The geometric consistency evaluation unit calculates the geometric error by comparing the point cloud deviation between the reconstructed model and the real scene. The texture detail analyzer evaluates the texture reconstruction quality by analyzing the image pixel distribution. The depth error calculator quantifies the depth prediction deviation by comparing the depth map.

[0023] In the urban high-end community building zoning modeling implementation example, based on existing urban planning data, remote sensing images, and population activity heat maps, the area is divided into high-density structural zones and medium-low-density structural zones according to factors such as building density, functional use, and traffic flow. A high-resolution wide-angle camera, RTK positioning module, and inertial measurement unit (IMU) mounted on a UAV are used to record camera pose and geographic coordinates in real time. All acquired image data and flight data undergo preliminary preprocessing through an edge computing gateway and are uploaded to the central modeling server. A sparse 3D point cloud is constructed based on simultaneous localization (SLAM) technology and image registration algorithms, and combined with the sparse image set to form an initial scene structure model. This sparse point cloud not only provides spatial geometric priors for subsequent modeling but also assists in identifying areas difficult to reconstruct and occlusion blind spots, automatically generating feedback maps of densely acquired suggested areas. Multi-view images (including sparse and dense areas) and the sparse point cloud are input into the NeRF training framework to perform volumetric modeling and rendering prediction. During training, the model employs a region-weighted strategy, increasing the image sampling frequency and training weights within densely populated areas to enhance its ability to handle complex structures like multi-story buildings. It incorporates spatial semantic labels, such as those for residential buildings, commercial buildings, green space boundaries, and parking lots. These labels are then fused with the NeRF model output through an external GIS system interface to create a 3D reconstruction with architectural semantic annotations. If the error in a high-density area exceeds the system's tolerance threshold, the system will feed back to the flight control unit, invoking a particle swarm optimization algorithm to replan the local acquisition path, re-capture the problematic area, and integrate it into the existing training set for retraining and updating. The NeRF method used in this embodiment does not require physical proximity to buildings or dedicated scanning equipment, allowing for rapid deployment under uninterrupted and normal conditions. It offers flexible acquisition, strong self-updating capabilities, and high output accuracy, making it particularly suitable for 3D digital twin modeling of large urban areas.

[0024] Specifically, the neural radiation field model is trained using a sparse image set. It takes pixel features from multi-view photographs and sparse point clouds as input, and bidirectionally models to obtain the spatial context features of the scene. This is used to reconstruct the 3D geometry and texture details of the captured scene. The formula for calculating the color of the neural radiation field volume rendering is as follows: , Where C^(r) is the predicted pixel color corresponding to the ray, N is the number of sampling points on the ray, σi is the volume density of the i-th sampling point, δi is the distance between the i-th point and the (i+1)-th point, and ci is the color of the i-th point in the viewing direction. The training loss function for neural radiation fields is: , Where Lcolor is the color regression loss, C^(r) is the model predicted color, Cgt(r) is the real image pixel color, and R is the set of rays sampled in the current training batch.

[0025] Specifically, the synchronous positioning technology analyzes the geometric relationships between the multi-view images, uses triangulation to calculate the three-dimensional spatial position of feature points, and constructs an initial sparse point cloud based on the reprojection error.

[0026] Specifically, the density-sparse path is dynamically balanced between coverage, energy consumption and reconstruction accuracy through a multi-objective optimization strategy based on the UAV's current flight attitude, remaining battery power, wind speed disturbance and flight safety boundary, to generate a set of density-sparse acquisition paths that meet the task constraints.

[0027] Specifically, the image acquisition module also includes an image quality assessment unit, which is used to automatically filter out overexposed, blurry, and highly occluded images after image acquisition, and to remove low-quality images before neural radiation field training.

[0028] In the museum artifact exhibition area embodiment, with the museum's authorization, a small drone equipped with a high-resolution visible light camera, an IMU module, and an RTK-GPS positioning system is used to pre-sample flight paths based on the exhibition area's structural information. According to the density of artifact distribution and the division of important exhibition areas, the control system automatically divides the entire exhibition area into "sparse acquisition areas" (such as aisles and open areas) and "dense acquisition areas" (such as glass display cases and sculpture stands). In sparse areas, the flight path is set in a regular grid pattern, acquiring one image per meter; in dense areas, a spiral orbit flight or multi-angle low-altitude hovering shooting mode is used, acquiring one image every 0.3 meters, and recording the camera attitude and synchronous positioning information. All acquired images are aligned with the RTK-GPS and IMU timestamps and sent to the synchronous positioning and sparse point cloud generation module. This module, based on the Image Feature Matching and Structure-from-Motion (SfM) algorithm, constructs a sparse 3D point cloud for the entire exhibition area and performs pose registration with each image. During the NeRF training phase, preprocessed multi-view images (including sparse and dense image sets) and their corresponding sparse point clouds are fed into the neural radiation field model. The model employs a bidirectional contextual modeling structure: by inputting the spatial location, viewing direction, and image features of each sampling point, it predicts its volume density and color, and calculates the final pixel color on the light rays based on the NeRF volume rendering integral formula. During training, a region weighting mechanism is introduced to increase the sampling frequency of densely sampled images by approximately three times, thereby enhancing the model's reconstruction accuracy in key detail areas (such as artifact edges and curved surface textures). After training, an automatic reconstruction quality assessment is performed through an error evaluation module. Evaluation metrics include geometric consistency (spatial error between the reconstructed point cloud and the actual measurement model), texture fidelity (image structural similarity), and depth consistency (error between the NeRF-generated depth map and the laser scan). If any indicator fails to meet the accuracy threshold (e.g., point cloud RMSE > 1cm), the system will trigger a drone to re-shoot, performing intensive localized photography of error hotspots and updating the model. This data will then be presented to viewers via a web interface or VR terminal, providing a rotatable, zoomable, and freely viewable 3D exhibition experience. Simultaneously, the model data is uploaded to the cultural relics digital archive cloud platform as a permanent digital copy, achieving efficient and high-precision 3D reconstruction of the museum's cultural relics environment.

[0029] Specifically, the neural radiation field model training process introduces a region sampling weight mechanism, which assigns a higher training frequency to image regions with high acquisition density.

[0030] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for improving the accuracy of the NERF algorithm by using densely packed drones to acquire images, characterized in that, include: S1: Based on the image acquisition module on the drone, multi-view images are acquired in the target scene using a preset density path, and an initial sparse point cloud is constructed through synchronous positioning technology; S2: Based on the multi-view images and sparse point clouds, call the sparse image set to train the neural radiation field model, generate the three-dimensional scene reconstruction result according to the multi-view images, and evaluate the reconstruction accuracy; S3: Based on the reconstruction accuracy assessment results, combined with the UAV flight status and scene complexity, the density is adjusted by planning the UAV flight path using the particle swarm algorithm to generate the best acquisition scheme for the current scene. S4: Analyze the optimal acquisition scheme through the quality reconstruction evaluation module, and perform detection based on evaluation indicators such as geometric consistency, texture detail, and depth error; if the reconstruction accuracy is found to be below expectations, automatically adjust the density and neural radiation field model parameters of the UAV.

2. The method according to claim 1, characterized in that, The image acquisition module includes a high-resolution camera, a synchronous positioning module, and an attitude calibration unit. The high-resolution camera is used to capture multi-view images of the target scene. The synchronous positioning module constructs a scene point cloud through synchronous positioning technology. The attitude calibration unit corrects the camera shooting angle through inertial measurement data.

3. The method according to claim 1, characterized in that, The sparse point cloud restores the spatial structure by fusing depth information between multi-view images, providing initial geometric constraints for the neural radiation field model and reducing convergence time in the early stages of training.

4. The method according to claim 1, characterized in that, The particle swarm optimization algorithm uses scene complexity and reconstruction accuracy as objective functions, and adjusts the drone's flight altitude and shooting angle by analyzing the geometric and texture complexity of the scene.

5. The method according to claim 1, characterized in that, The quality reconstruction evaluation module includes a geometric consistency evaluation unit, a texture detail analyzer, and a depth error calculator. The geometric consistency evaluation unit calculates the geometric error by comparing the point cloud deviation between the reconstructed model and the real scene. The texture detail analyzer evaluates the texture reconstruction quality by analyzing the pixel distribution of the image. The depth error calculator quantifies the depth prediction deviation by comparing the depth map.

6. The method according to claim 3, characterized in that, The neural radiation field model is trained using a sparse image set. By taking the pixel features of multi-view photos and sparse point clouds as input, it bidirectionally models and obtains the spatial context features of the scene, which are used to reconstruct the three-dimensional geometry and texture details of the acquired scene.

7. The method according to claim 4, characterized in that, The synchronous positioning technology analyzes the geometric relationships between the multi-view images, uses triangulation to calculate the three-dimensional spatial position of feature points, and constructs an initial sparse point cloud based on the reprojection error.

8. The method according to claim 1, characterized in that, The density-sparse path is dynamically balanced between coverage, energy consumption and reconstruction accuracy through a multi-objective optimization strategy based on the UAV's current flight attitude, remaining battery power, wind speed disturbance and flight safety boundary, generating a density-sparse acquisition path set that meets the task constraints.

9. The method according to claim 2, characterized in that, The image acquisition module also includes an image quality assessment unit, which is used to automatically filter out overexposed, blurry, and highly occluded images after image acquisition, and to remove low-quality images before neural radiation field training.

10. The method according to claim 4, characterized in that, The neural radiation field model training process introduces a regional sampling weight mechanism, which assigns a higher training frequency to image regions with high acquisition density.

Images