Building three-dimensional reconstruction method based on 3D Gaussian sputtering guidance

Abstract

The invention discloses a building three-dimensional reconstruction method based on 3D Gaussian sputtering guidance. The building three-dimensional reconstruction method comprises the following steps: S1, acquiring multi-view building image data; s2, sparse point cloud reconstruction of the building is carried out from the multi-view building image based on a motion recovery structure SFM, and then the point cloud of the building is initialized through 3D Gaussian Splitting; s3, performing multi-scale optimization representation on the initialized 3D Gaussian point cloud, accurately capturing local geometric features and performing optimization adjustment on surface fitness so as to enhance representation of a building surface; s4, building patches are generated based on three-dimensional Gaussian distribution; and S5, obtaining a building three-dimensional model and a rendering result. According to the scheme, a highly detailed surface can be recovered, and high-precision and high-fidelity reconstruction of the geometric shape and appearance of a building in a complex scene is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a three-dimensional reconstruction method of a building based on 3D Gaussian sputtering guidance. Background Art

[0002] Three-dimensional (3D) reconstruction of buildings refers to obtaining a 3D geometric model of a building structure by scanning the building using sensors such as RGB cameras and depth cameras and collecting spatial information. In recent years, achieving high-quality 3D reconstruction of buildings under complex conditions has become a research focus in the field of photogrammetry and computer vision. High-precision 3D modeling of building structures has important application value in many fields such as urban planning, disaster simulation, and cultural heritage protection.

[0003] Since the beginning of the 21st century, there has been an increasing number of methods for architectural reconstruction. Traditional 3D reconstruction methods, including multi-view stereo (MVS), structured light reconstruction, and image-based modeling techniques, have achieved remarkable success in simple scenes. However, these methods often face many challenges when applied to scenes containing complex structures, such as buildings. Such scenes usually include a large amount of vegetation and roads, with complex structures, significant lighting changes, and diverse perspectives, which bring great difficulties to traditional 3D reconstruction methods.

[0004] With the rapid development of Neural Radiance Field (NeRF) technology, traditional 3D reconstruction methods have made breakthrough progress. Unlike traditional explicit reconstruction techniques, NeRF generates a continuous 3D scene representation by optimizing a set of directional images, thereby simulating the free expression of 3D space at the image level. NeRF is able to generate imaging-quality scene reconstruction effects, overcoming the visual limitations of traditional 3D reconstruction methods. Although NeRF is based on a self-supervised learning model, its training process usually takes hours, limiting its practicality.

[0005] The emergence of 3D Gaussian Splatting (3D GS) has brought new hope and changes to the high-precision reconstruction of buildings in complex scenes. The 3D GS method uses Gaussian functions for scene representation, which enhances its adaptability in depicting complex scenes and shortens the training time from hours to minutes. The main advantages of this method over traditional technologies and NeRF are: 1. The Gaussian representation of 3D GS is naturally suitable for complex scenes and can achieve real-time and realistic rendering of realistic scenes. 2. It has an extremely fast training speed, and preliminary rendering results can be obtained in about five minutes. 3. Unlike the implicit representation of NeRF, the 3D GS method uses explicit expression, which can directly retrain and optimize the initialized Gaussian points. However, although 3D GS can achieve realistic rendering of new perspectives in complex scenes, the distribution of Gaussian points may not fully conform to the surface of the building and cannot meet the precise measurement requirements of detailed surfaces. Summary of the invention

[0006] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a new method for complex building reconstruction based on images and guided by 3D Gaussian scattering to restore highly detailed surfaces and achieve high-precision and high-fidelity reconstruction of building geometry and appearance in complex scenes.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is: a three-dimensional reconstruction method of a building based on 3D Gaussian sputtering guidance, comprising the following steps:

[0008] S1, obtaining multi-view building image data;

[0009] S2, reconstructing the sparse point cloud of the building from the multi-view building image based on the motion recovery structure SFM, and then initializing the building point cloud through 3D Gaussian Splatting;

[0010] S3, perform multi-scale optimization representation on the initialized 3D Gaussian point cloud, accurately capture local geometric features, and optimize the surface fit, thereby enhancing the representation of the building surface;

[0011] S4, generating building patches based on three-dimensional Gaussian distribution;

[0012] S5. Obtain a three-dimensional building model and rendering results.

[0013] In step S1, a camera mounted on a drone is used to capture multi-angle images of a building to obtain multi-view building image data.

[0014] Step S2 includes: the structure from motion SFM first extracts feature points from the building image and matches the feature points in images of different viewing angles;

[0015] When the feature points are matched, the corresponding position and posture of each image are used to calculate the three-dimensional coordinates of the matching feature points through triangulation. Then, the camera parameters and three-dimensional point positions are globally optimized by minimizing the reprojection error, thereby obtaining a sparse point cloud of the building and the camera pose that can be used as model input.

[0016] The obtained sparse point cloud is processed based on 3D Gaussian Splatting, including: 3D Gaussian Splatting converts the sparse three-dimensional point cloud into a continuous, Gaussian distribution state, places a three-dimensional Gaussian distribution at the position of each point cloud data point, thereby achieving smoothing and completion of the sparse point cloud, and forming an initial three-dimensional Gaussian point cloud model.

[0017] When converting the sparse point cloud into the initial 3D Gaussian point cloud, the distribution of the initial Gaussian point cloud is expressed according to the following formula:

[0018]

[0019] Among them, N(x;u i ,∑ i ) represents the Gaussian distribution at position x, u i is the mean vector of the Gaussian distribution, ∑ i is the covariance matrix. The Gaussian distribution ensures the smooth transition and continuity of the point cloud in three-dimensional space. i The shape and expansion range of the Gaussian distribution are controlled, and the shape of the Gaussian point cloud is changed to adapt to different point cloud densities and distributions.

[0020] In step S3, optimizing the building point cloud using three-dimensional Gaussian point cloud optimization includes:

[0021] (1) Multi-scale Gaussian representation and fusion

[0022] The Gaussian parameters {μ s ,∑ s} Optimized for a specific level of detail, the multi-scale fusion strategy is mathematically described as the following weighted averaging process: Define the weight w of each scale s s , weights are assigned according to the importance or contribution of the scale, and Gaussian parameters of different scales are fused to form the final scene representation:

[0023] G final =Σ s w s ·G s

[0024] Among them, G s ={μ s ,Σ s} is the Gaussian parameter set of the sth scale;

[0025] (2) Regularization term based on local curvature optimization

[0026] In order to refine the Gaussian distribution point cloud in the process of 3D reconstruction of buildings and make the Gaussian points better adapt to the surface geometry of the scene, a regularization term based on local curvature optimization is introduced. This regularization term constrains the curvature change and focuses on punishing unnecessary curvature or noise, thereby obtaining a smoother and more accurate model; the regularization objective function is defined as:

[0027] E reg =∑ i (||∑ i -∑ target (κ i )|| 2 )

[0028] Among them, Σ tar get (k i ) represents the target covariance matrix based on the local curvature. The local curvature κ can be calculated by the following method:

[0029]

[0030] Where n is the normal vector of the point cloud.

[0031] (3) 3D Gaussian retraining for building surface fitting

[0032] Three-dimensional Gaussian retraining is performed on the surface of the building. During the retraining process, the signed distance function SDF is first used to fit the building surface. SDF can accurately describe the distance and direction from a point to the surface. By optimizing SDF, a smooth and accurate surface representation is obtained, which provides a basis for subsequent steps. On this fitted surface, the optimization of the density function is introduced. By adjusting the density function to increase the overlap of the three-dimensional Gaussians, the distribution of different Gaussians on the surface of the building is ensured to be more uniform and reasonable, thereby improving the effect of three-dimensional Gaussian retraining and making it better fit the surface characteristics of the building. Finally, the three-dimensional Gaussian ellipsoid is flattened by reducing the three-dimensional Gaussian scaling factor to adjust the shape of the Gaussian ellipsoid to make it flatter and fit the surface structure of the building. By reducing the scaling factor, the accuracy and effect of three-dimensional Gaussian retraining are improved.

[0033] The method for generating building facets based on three-dimensional Gaussian distribution in step S4 includes: a building surface mesh initial reconstruction step and a three-dimensional Gaussian mesh joint optimization step.

[0034] The initial reconstruction step of the building surface mesh includes obtaining a preliminary point cloud to capture the complex geometric structure and details of the building after completing the 3D Gaussian point optimization of the building; resampling from the optimized Gaussian points to ensure the uniform and reasonable distribution of the point cloud data, so as to better represent the surface characteristics of the object; the resampling of the optimized Gaussian points takes into account the changes in the point cloud density function ρ(x) and adaptively adjusts the sampling rate λ(x):

[0035]

[0036] Among them, λ min and λ max are the minimum and maximum sampling rate thresholds, respectively, th is the density threshold; subsequently, the Poisson reconstruction method is used to generate the initial triangular mesh, and a global least square solution is constructed to smooth the point cloud data to generate a continuous and consistent triangular mesh surface. The divergence operator div is used to calculate the divergence of the vector field v generated by the optimized Gaussian points. The vector field v represents the direction and intensity distribution of the point cloud and is the key to describing the surface characteristics of the building.

[0037] Then use the Laplace operator Δ to calculate the second-order derivative of the function f by solving the Poisson equation:

[0038] Δf=div v

[0039] A continuous function f consistent with the point cloud data is obtained, and its solution constitutes the initial mesh surface of the building.

[0040] The steps of joint optimization of three-dimensional Gaussian grid include: using SDF to refine and optimize the three-dimensional grid, SDF defines a distance value for each spatial point, which represents the distance from the point to the nearest surface, where a positive value indicates that the point is outside the object and a negative value indicates that the point is inside the object; for each grid point p, calculate its SDF value SDF(p):

[0041] SDF(p)=sign(p)·min q∈Surface ||pq||

[0042] Where sign(p) is the sign function, p is a point in the mesh, q is the surface point closest to p, and -1 or +1 is returned depending on whether point p is inside or outside the surface. The mesh is refined based on the SDF value, and the mesh quality and detail representation are optimized by inserting or deleting vertices. If |SDF(p)|>θ, point p needs to be moved to better fit the surface. The continuity of the mesh and the smoothness of the surface are optimized by resampling and adjusting the local mesh.

[0043] The advantages of the present invention are: 1. 3D Gaussian scattering is used to guide the smoothing and completion of the sparse point cloud generated by SFM, creating a high-density point cloud, which significantly enhances the continuity and realism of the point cloud, thereby laying a solid foundation for the subsequent geometric reconstruction of the building.

[0044] 2. A three-dimensional Gaussian point cloud optimization strategy is designed for complex building surface structures. Multi-scale Gaussian representation and fusion technology is adopted, and combined with the local curvature optimization regularization term, the 3D Gaussian points are made to more accurately conform to the building surface geometry, thereby enhancing the ability to capture building surface details and improving the smoothness and visual consistency of the reconstructed model.

[0045] 3. Use the Poisson algorithm to optimize the surface of complex buildings to reconstruct the initial triangular mesh, and use Signed Distance Fields (SDF) to further optimize the geometric accuracy and surface detail representation, thereby improving the detail fidelity and overall reconstruction quality of the complex building surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The following is a brief description of the contents expressed in the drawings of the present invention and the symbols in the drawings:

[0047] Figure 1 The SFM sparse point cloud generation flow chart of the present invention;

[0048] Figure 2 The point cloud images generated by the present invention; (a) is the sparse point cloud generated by SFM; (b) is the point cloud initialized by 3D Gaussian. DETAILED DESCRIPTION

[0049] The specific implementation of the present invention will be further explained in detail below by describing the optimal embodiment with reference to the accompanying drawings.

[0050] This paper proposes a new image-based complex building reconstruction method guided by 3D Gaussian scattering to restore highly detailed surfaces and achieve high-precision and high-fidelity reconstruction of building geometry and appearance in complex scenes. The main contributions of this paper include:

[0051] 1. 3D Gaussian scattering is used to guide the smoothing and completion of the sparse point cloud generated by SFM to create a high-density point cloud, which significantly enhances the continuity and realism of the point cloud, thus laying a solid foundation for the subsequent geometric reconstruction of the building.

[0052] 2. A three-dimensional Gaussian point cloud optimization strategy is designed for complex building surface structures. Multi-scale Gaussian representation and fusion technology is adopted, and combined with the local curvature optimization regularization term, the 3D Gaussian points are made to more accurately conform to the building surface geometry, thereby enhancing the ability to capture building surface details and improving the smoothness and visual consistency of the reconstructed model.

[0053] 3. Use the Poisson algorithm to optimize the surface of complex buildings to reconstruct the initial triangular mesh, and use Signed Distance Fields (SDF) to further optimize the geometric accuracy and surface detail representation, thereby improving the detail fidelity and overall reconstruction quality of the complex building surface.

[0054] like Figure 1 , 2 As shown, the technical solution adopted by the present invention is: a new method for complex building reconstruction based on images and guided by 3D Gaussian scattering includes the following steps:

[0055] S1, acquisition of multi-view building image dataset;

[0056] S2. 3D Gaussian point cloud initialization: Based on the motion recovery structure (SFM), the sparse point cloud of the building is reconstructed from the multi-view building images, and then the building point cloud is initialized by 3D Gaussian Splatting;

[0057] S3. Optimization of 3D Gaussian point cloud for buildings: Multi-scale optimization representation of the initialized 3D Gaussian point cloud, accurate capture of local geometric features, and optimization adjustment of surface fit are performed to enhance the representation of the building surface.

[0058] S4. Building patch generation method based on three-dimensional Gaussian distribution;

[0059] S5. Obtain the three-dimensional model of the building and the rendering result; the patch is the core part of building a three-dimensional model, but simply generating the patch is not enough to form a complete three-dimensional model. The patches need to be connected according to topological rules to form a closed or continuous surface structure (three-dimensional model of the building). The three-dimensional model of the building is generated by extracting patches from the optimized Gaussian point cloud using the Poisson algorithm. The patch itself does not contain visual information such as texture and lighting, and the rendering result needs to be obtained through texture mapping. Among them:

[0060] For texture extraction: extract texture information from multi-view images and calculate the UV texture coordinates corresponding to each patch.

[0061] Map to patches: Map texture images onto patches to provide color and material information for the model.

[0062] In step S1, the multi-view building image dataset consists of the Small Buildings dataset provided by ArcGIS and the Tower dataset collected by drones. When verifying this algorithm, public datasets and self-measured datasets can be used. When using this method to reconstruct buildings, it is necessary to collect image data from multiple angles of the building to be reconstructed through various means such as drones.

[0063] In step S2, the SFM algorithm first extracts feature points from the building image and matches the feature points in images of different viewpoints. After the feature points are matched, the three-dimensional coordinates of the matching feature points are calculated by triangulation using the corresponding position and posture of each image. Then, the camera parameters and three-dimensional point positions are globally optimized by minimizing the reprojection error, thereby obtaining a sparse point cloud of the building and a camera pose that can be used as model input. Subsequently, the obtained sparse point cloud is processed based on 3D Gaussian Splatting. 3D Gaussian Splatting can convert sparse three-dimensional point clouds into a continuous, Gaussian distribution state. The basic principle is to place a three-dimensional Gaussian distribution at the position of each point cloud data point, thereby achieving smoothing and completion of the sparse point cloud to form an initial 3D Gaussian point cloud.

[0064] Optimizing parameters and positions by minimizing the reprojection error includes: The sparse point cloud and camera pose results directly affect the accuracy of subsequent Gaussian reconstruction. The sparse point cloud and camera pose are optimized by minimizing the error function G(C, X) and the model accuracy is evaluated.

[0065]

[0066] where ρ ij represents the indicator function; q ij represents the observed image point; C i and X j Represent the camera parameters and 3D point coordinates respectively; P(C i , X j ) represents the projection point of the 3D point under the camera’s view. By minimizing the error function, the SFM algorithm refines the camera parameters and 3D point positions to produce an accurate sparse point cloud representation.

[0067] In this method, the sparse point cloud is converted into an initial 3D Gaussian point cloud, and the distribution of the initial Gaussian point cloud is expressed according to the following formula:

[0068]

[0069] Among them, N(x;u i ,Σ i ) represents the Gaussian distribution at position x, ui is the mean vector of the Gaussian distribution, Σ i is the covariance matrix. The Gaussian distribution ensures the smooth transition and continuity of the point cloud in three-dimensional space. i Controls the shape and expansion range of the Gaussian distribution. The shape of the Gaussian point cloud can be changed to adapt to different point cloud densities and distributions. The letter T represents the transpose of a vector, which is a mathematical operator. i ) T is a column vector (xu i ) to turn it into a row vector.

[0070] In step S3, the three-dimensional Gaussian point cloud optimization of the building is applied, wherein the optimization scheme includes:

[0071] (1) Multi-scale Gaussian representation and fusion

[0072] When dealing with complex objects such as buildings, the multi-scale Gaussian representation can better capture the details of the building from macro to micro, and the Gaussian parameters {μ s ,∑ s} Optimize for a specific level of detail. The multi-scale fusion strategy can be mathematically described as the following weighted averaging process: Define the weight w of each scale s s , these weights are assigned according to the importance or contribution of the scale, and the Gaussian parameters of different scales are fused to form the final scene representation. As shown in the following formula:

[0073] G final =∑ s w s ·G s

[0074] Among them, G s ={μ s ,∑ s} is the Gaussian parameter set of the sth scale. This process ensures the richness of details and the naturalness of visual effects during reconstruction, and can effectively balance the preservation of details and the overall consistency of the scene.

[0075] (2) Regularization term based on local curvature optimization

[0076] In order to further refine the Gaussian distribution point cloud in the process of 3D reconstruction of buildings and make the Gaussian points better adapt to the surface geometry of the scene, a regularization term based on local curvature optimization is introduced. This term constrains the curvature change and focuses on punishing unnecessary curvature or noise, thereby obtaining a smoother and more accurate model. The regularization objective function is defined as:

[0077] E reg =∑ i (||∑i -∑ target (κ i )|| 2 )

[0078] Among them, ∑ target (κ i ) represents the target covariance matrix based on the local curvature. The local curvature κ can be calculated by the following method:

[0079]

[0080] Here, n is the normal vector of the point cloud. The local curvature reflects the deformation characteristics of the point cloud in a small range, effectively reducing the errors caused by noise and data inconsistency, thereby improving the geometric accuracy and visual effect of the reconstructed scene to a greater extent. This adjustment can enhance the details and realism of the 3D reconstruction, and to a certain extent, it can improve the ability of the method to handle complex architectural features, ensuring high-quality reconstruction results.

[0081] The operations required to introduce the regularization term are:

[0082] (1) Calculate the local curvature of the point cloud

[0083]

[0084] (2) Define the regularization term

[0085] The local curvature value is used as the core parameter of the regularization constraint to construct the target covariance matrix to restrict the local geometry of the points and ensure the smoothness of the transition.

[0086] E reg =∑ i (||∑ i -∑ target (κ i )|| 2 )(3) 3D Gaussian retraining for building surface fitting

[0087] Although the improved 3D GS method can obtain high-fidelity 3D building effects after training point distribution, there is currently no method that can directly measure from Gaussian points. The method directly based on Gaussian points cannot generate a triangular mesh that fits the building surface when patching, so 3D Gaussian retraining is required for the building surface. (1) and (2) in step S3 are synchronous optimizations of the point cloud, and (3) retraining is performed on the optimized point cloud.

[0088] In the retraining process, the signed distance function (SDF) is first used to fit the building surface. SDF can accurately describe the distance and direction from a point to the surface. By optimizing the SDF, a smooth and accurate surface representation can be obtained, thus providing a basis for subsequent steps. On this fitted surface, the optimization of the density function is introduced. By adjusting the density function to increase the overlap of the three-dimensional Gaussians, the distribution of different Gaussians on the building surface is ensured to be more uniform and reasonable, thereby improving the effect of the three-dimensional Gaussian retraining and making it better fit the surface features of the building. Finally, the three-dimensional Gaussian ellipsoid is flattened by reducing the three-dimensional Gaussian scaling factor to adjust the shape of the Gaussian ellipsoid to make it flatter and fit the surface structure of the building. By reducing the scaling factor, the accuracy and effect of the three-dimensional Gaussian retraining can be further improved.

[0089] In the retraining process, the signed distance function (SDF) is first used to fit the building surface. SDF can accurately describe the distance from a point to the surface and its direction. The formula is:

[0090]

[0091] Among them, d(x, S) represents the Euclidean distance from point x to surface S. By optimizing SDF, a smooth and accurate surface representation can be obtained, which provides a basis for subsequent steps.

[0092] On this fitted surface, the optimization of the density function is introduced. By adjusting the density function ρ(x) to increase the overlap of the three-dimensional Gaussians, the distribution of different Gaussians on the building surface is ensured to be more uniform and reasonable. The density function optimization can be expressed as:

[0093]

[0094] Among them, α i is the weight of the i-th Gaussian point, x i is the position of the Gaussian point, σ is the standard deviation of the Gaussian function, and by adjusting α i and σ can control the density distribution, thereby optimizing the overlap and improving the effect of 3D Gaussian retraining to make it better fit the surface characteristics of the building.

[0095] Finally, the three-dimensional Gaussian ellipsoid is flattened by reducing the three-dimensional Gaussian scaling factor to adjust the shape of the Gaussian ellipsoid to make it flatter and fit the surface structure of the building. The shape of the Gaussian ellipsoid is usually described by the scaling factor s and the main axis lengths λ1, λ2, λ3, and the formula is:

[0096]

[0097] Where x = (x1, x2, x3) is an arbitrary point in three-dimensional space. By adjusting s, the Gaussian ellipsoid can be made flatter, so that it fits the building surface better. By optimizing the scaling factor, the accuracy and effect of three-dimensional Gaussian retraining can be further improved.

[0098] In step S4, the method for generating building facets based on three-dimensional Gaussian distribution specifically includes: (In this step, facets and grids are the same concept, and grids are used in this step.)

[0099] (1) Initial reconstruction of building surface mesh

[0100] After completing the optimization of the 3D Gaussian point cloud of the building, a preliminary point cloud is obtained that can be used to capture the complex geometric structure and details of the building. Next, resampling is performed from the optimized Gaussian point cloud to ensure a uniform and reasonable distribution of the point cloud data, so as to better represent the surface features of the object, which is essential for generating an accurate triangulated mesh. Optimize the resampling of the Gaussian point cloud. Considering the change of the point cloud density function ρ(x), the sampling rate λ(x) is adaptively adjusted:

[0101]

[0102] Among them, λ min and λ max are the minimum and maximum sampling rate thresholds, respectively, th The adaptive sampling strategy effectively reduces the number of sampling points in high-density areas to avoid oversampling, and increases the number of sampling points in low-density areas to capture finer details.

[0103] For the point cloud data obtained after resampling, we use the Poisson reconstruction method to generate the initial triangular mesh. This method smoothes the point cloud data by constructing a global least squares solution to generate a continuous and consistent triangular mesh surface. The divergence operator div is used to calculate the divergence of the vector field v generated by the optimized Gaussian points. The vector field v represents the direction and intensity distribution of the point cloud and is the key to describing the surface characteristics of the building.

[0104] Then use the Laplace operator Δ to calculate the second-order derivative of the function f by solving the Poisson equation:

[0105] Δf=div v

[0106] We can obtain a continuous function f that is consistent with the point cloud data, and its solution constructs the initial mesh surface of the building. This method ensures the smoothness and consistency of the mesh while maintaining the accurate expression of geometric details.

[0107] The Poisson reconstruction method is known for its robustness and efficiency. It can effectively process complex surface shapes and generate high-quality triangular meshes. Compared with other reconstruction methods, Poisson reconstruction performs well in dealing with noise and non-uniformly sampled point clouds, and can better preserve the details and complex structures of objects.

[0108] (2) Joint optimization of three-dimensional Gaussian grids

[0109] The Poisson reconstruction method is widely used to generate the initial 3D mesh, however, it has obvious limitations. First, Poisson reconstruction tends to generate smooth surfaces, which may lead to loss of details, especially when dealing with objects with complex surface features. In addition, in areas with too much filled or missing data, Poisson reconstruction may generate unrealistic surfaces, resulting in inaccurate edges, especially at boundaries or irregular edges, thus affecting the overall quality of the 3D model.

[0110] To address these issues, SDF is used to refine and optimize the mesh. The SDF method enhances the processing capabilities of the original point cloud details, provides better noise resistance, and is more efficient in processing large-scale data. Compared with Poisson reconstruction, SDF performs well in filling data gaps and generating realistic surfaces, while producing accurate edges at open or irregular boundaries, significantly improving the overall quality of the 3D model.

[0111] The initialized 3D network obtained by Poisson reconstruction is further refined and optimized through SDF. SDF is used to further refine and optimize the 3D mesh. SDF defines a distance value for each spatial point, which represents the distance from the point to the nearest surface, where a positive value indicates that the point is outside the object and a negative value indicates that the point is inside the object. For each grid point p, calculate its SDF value SDF(p):

[0112] SDF(p)=sign(p)·min q∈Surface ||pq||

[0113] Where sign(p) is a function, p is a point in the mesh, q is the surface point closest to p, and returns -1 or +1 depending on whether point p is inside or outside the surface. The mesh is refined based on the SDF value, and the mesh quality and detail representation are optimized by inserting or deleting vertices. If |SDF(p)|>θ, point p needs to be moved to better fit the surface. θ here represents a threshold parameter used to determine whether the mesh point p needs to be adjusted. The continuity of the mesh and the smoothness of the surface are optimized by resampling and adjusting the local mesh.

[0114] Mesh refinement based on SDF values ​​includes:

[0115] 1. Insert new points to optimize mesh quality

[0116] When the mesh triangles in a certain area are too large or too sparse, new points need to be inserted to refine the local triangular mesh structure and maintain local smoothness.

[0117] 2. Move points to better fit the surface

[0118] When a grid point deviates from the target surface (i.e., the SDF value is close to but not zero), the point needs to be moved

[0119] 3. Delete points to optimize the mesh

[0120] When the point density in a certain area is too high and the distribution of points does not contribute significantly to the surface characteristics, redundant points need to be deleted.

[0121] 4. Resampling and adjustment of local grid

[0122] When the local mesh continuity or smoothness is poor (for example, the curvature changes sharply or the mesh topology is irregular), it is necessary to resample the mesh points.

[0123] Curvature Adaptive Sampling: Adjusts point density based on local curvature, adaptively increasing or decreasing sampling points according to the density function.

[0124] Mesh adjustment: Regenerate the local triangle mesh based on the resampled point cloud to ensure that the mesh shape is regular and the surface is smooth.

[0125] Experimental results analysis:

[0126] The reconstruction method provided by this scheme is compared with the traditional method colmap and the latest neural radiance field method neuralangelo. As a representative of the classical method, colmap relies on multi-view geometry (SFM) and multi-view stereo matching (MVS) to recover sparse point clouds and generate dense 3D reconstruction models. Similarly, we also used colmap in the initial process to recover sparse point clouds from images through SFM as the basis of the reconstruction method of this scheme. Neuralangelo performs coarse-to-fine optimization on the hash grid of each level of detail, reconstructs high-fidelity 3D surface structures from multi-view images, and achieves state-of-the-art results. The performance of each method in 3D model generation and rendering tasks is analyzed.

[0127] In order to quantitatively evaluate the performance of colmap, neuralangelo and the method proposed in this paper, PSNR and SSIM are used as evaluation indicators. After the network training is completed, a certain number of images are randomly selected from the image dataset for testing. According to the camera poses of these images, the corresponding rendered images are generated for the 3D model, and the PSNR and SSIM values ​​between the rendered images and the original images are calculated, and the average value is taken as the final overall evaluation result of each method. Table 1 compares the PSNR and SSIM values ​​of the three methods on the rendered images. The results show the advantages of the method of this scheme in terms of fidelity and structural consistency.

[0128] Table 1. Comparison of PSNR and SSIM values ​​of three methods

[0129]

[0130] As shown in Table 1, for the comparison of PSNR and SSIM values ​​of the two datasets, our proposed method significantly outperforms the colmap and neuralangelo methods in both PSNR and SSIM.

[0131] On the Small Buildings dataset, it can be observed that the neuralangelo method has the lowest PSNR and SSIM values, which are 17.31 and 0.76 respectively. This shows that the rendering effect of neuralangelo is far from the pixel value of the original image, there is a lot of noise and distortion, and the details in the original image cannot be effectively restored. In addition, the rendered image is significantly different from the original image in brightness, contrast and structural information, the structural similarity is relatively low, and the visual quality is poor. Although the colmap method has been improved on the basis of neuralangelo, it is still difficult to achieve the desired effect. In contrast, our method has far surpassed colmap and neuralangelo, reaching 29.67 and 0.94 in PSNR and SSIM values. This substantial improvement shows that our method provides more accurate surface reconstruction, which can effectively reduce noise and distortion while ensuring details, making the rendered image closer to the visual effect of the original image.

[0132] On the Tower dataset, colmap and neuralangelo methods have similar performance, with PSNR values ​​of 25.18 and 24.27, and SSIM values ​​of 0.91 and 0.87, respectively. However, our method reaches 32.69 and 0.96 in PSNR and SSIM values, indicating that our method is more effective in preserving the complex structural details and texture consistency present in the original image, especially in the fine-grained representation of building structures, which is crucial for complex building structures.

[0133] Obviously, the specific implementation of the present invention is not limited to the above-mentioned methods. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention, they are all within the protection scope of the present invention.

Claims

1. A method for 3D reconstruction of buildings based on 3D Gaussian sputtering guidance, characterized in that: The steps include: S1, obtaining multi-view building image data; S2, reconstructing the sparse point cloud of the building from the multi-view building image based on the motion recovery structure SFM, and then initializing the building point cloud through 3D Gaussian Splatting; S3, perform multi-scale optimization representation on the initialized 3D Gaussian point cloud, accurately capture local geometric features, and optimize the surface fit, thereby enhancing the representation of the building surface; S4, generating building patches based on three-dimensional Gaussian distribution; S5. Obtain a three-dimensional building model and rendering results.

2. The method for three-dimensional reconstruction of a building based on 3D Gaussian sputtering guidance according to claim 1, characterized in that: In step S1, a camera mounted on a drone is used to capture multi-angle images of a building to obtain multi-view building image data.

3. The method for three-dimensional reconstruction of a building based on 3D Gaussian sputtering guidance according to claim 1, characterized in that: Step S2 includes: the structure from motion SFM first extracts feature points from the building image and matches the feature points in images of different viewing angles; When the feature points are matched, the corresponding position and posture of each image are used to calculate the three-dimensional coordinates of the matching feature points through triangulation. Then, the camera parameters and three-dimensional point positions are globally optimized by minimizing the reprojection error, thereby obtaining a sparse point cloud of the building and the camera pose that can be used as model input.

4. The method for 3D reconstruction of a building based on 3D Gaussian sputtering guidance according to claim 3, characterized in that: The obtained sparse point cloud is processed based on 3D Gaussian Splatting, including: 3D Gaussian Splatting converts the sparse three-dimensional point cloud into a continuous, Gaussian distribution state, places a three-dimensional Gaussian distribution at the position of each point cloud data point, thereby achieving smoothing and completion of the sparse point cloud, and forming an initial three-dimensional Gaussian point cloud model.

5. The method for three-dimensional reconstruction of a building based on 3D Gaussian sputtering guidance according to claim 4, characterized in that: When converting the sparse point cloud into the initial 3D Gaussian point cloud, the distribution of the initial Gaussian point cloud is expressed according to the following formula: Among them, N(x;u i ,∑ i ) represents the Gaussian distribution at position x, u i is the mean vector of the Gaussian distribution, ∑ i is the covariance matrix. The Gaussian distribution ensures the smooth transition and continuity of the point cloud in three-dimensional space. i The shape and expansion range of the Gaussian distribution are controlled, and the shape of the Gaussian point cloud is changed to adapt to different point cloud densities and distributions.

6. A method for three-dimensional reconstruction of a building based on 3D Gaussian sputtering guidance according to any one of claims 1 to 5, characterized in that: In step S3, optimizing the building point cloud using three-dimensional Gaussian point cloud optimization includes: (1) Multi-scale Gaussian representation and fusion The Gaussian parameters {μ s ,∑ s } Optimized for a specific level of detail, the multi-scale fusion strategy is mathematically described as the following weighted averaging process: Define the weight w of each scale s s , weights are assigned according to the importance or contribution of the scale, and Gaussian parameters of different scales are fused to form the final scene representation: G final =∑ s w s ·G s Among them, G s ={μ s ,∑ s } is the Gaussian parameter set of the sth scale; (2) Regularization term based on local curvature optimization In order to refine the Gaussian distribution point cloud in the process of 3D reconstruction of buildings and make the Gaussian points better adapt to the surface geometry of the scene, a regularization term based on local curvature optimization is introduced. This regularization term constrains the curvature change and focuses on punishing unnecessary curvature or noise, thereby obtaining a smoother and more accurate model; the regularization objective function is defined as: E reg =∑ i (||∑ i -∑ target (k i )|| 2 ) Among them, ∑ target (κ i ) represents the target covariance matrix based on the local curvature. The local curvature κ can be calculated by the following method: Where n is the normal vector of the point cloud. (3) 3D Gaussian retraining for building surface fitting Three-dimensional Gaussian retraining is performed on the surface of the building. During the retraining process, the signed distance function SDF is first used to fit the building surface. SDF can accurately describe the distance and direction from a point to the surface. By optimizing SDF, a smooth and accurate surface representation is obtained, which provides a basis for subsequent steps. On this fitted surface, the optimization of the density function is introduced. By adjusting the density function to increase the overlap of the three-dimensional Gaussians, the distribution of different Gaussians on the surface of the building is ensured to be more uniform and reasonable, thereby improving the effect of three-dimensional Gaussian retraining and making it better fit the surface characteristics of the building. Finally, the three-dimensional Gaussian ellipsoid is flattened by reducing the three-dimensional Gaussian scaling factor to adjust the shape of the Gaussian ellipsoid to make it flatter and fit the surface structure of the building. By reducing the scaling factor, the accuracy and effect of three-dimensional Gaussian retraining are improved.

7. A method for 3D reconstruction of a building based on 3D Gaussian sputtering guidance according to any one of claims 1 to 5, characterized in that: The method for generating building facets based on three-dimensional Gaussian distribution in step S4 includes: a building surface mesh initial reconstruction step and a three-dimensional Gaussian mesh joint optimization step.

8. The method for three-dimensional reconstruction of a building based on 3D Gaussian sputtering guidance according to claim 7, characterized in that: The initial reconstruction step of the building surface mesh includes obtaining a preliminary point cloud to capture the complex geometric structure and details of the building after completing the 3D Gaussian point optimization of the building; resampling from the optimized Gaussian points to ensure the uniform and reasonable distribution of the point cloud data, so as to better represent the surface characteristics of the object; The optimized resampling of Gaussian points takes into account the change of the point cloud density function ρ(x) and adaptively adjusts the sampling rate λ(x): Among them, λ min and λ max are the minimum and maximum sampling rate thresholds, respectively, th is the density threshold; subsequently, the Poisson reconstruction method is used to generate the initial triangular mesh, and a global least square solution is constructed to smooth the point cloud data to generate a continuous and consistent triangular mesh surface. The divergence operator div is used to calculate the divergence of the vector field v generated by the optimized Gaussian points. The vector field v represents the direction and intensity distribution of the point cloud and is the key to describing the surface characteristics of the building. Then use the Laplace operator Δ to calculate the second-order derivative of the function f by solving the Poisson equation: Δf=div v A continuous function f consistent with the point cloud data is obtained, and its solution constitutes the initial mesh surface of the building.

9. A method for three-dimensional reconstruction of a building based on 3D Gaussian sputtering guidance according to any one of claims 1 to 5, characterized in that: The steps of joint optimization of three-dimensional Gaussian grid include: using SDF to refine and optimize the three-dimensional grid, SDF defines a distance value for each spatial point, which represents the distance from the point to the nearest surface, where a positive value indicates that the point is outside the object and a negative value indicates that the point is inside the object; for each grid point p, calculate its SDF value SDF(p): SDF(p)=sign(p)·min q∈Surface ||p-q|| Where sign(p) is the sign function, p is a point in the mesh, q is the surface point closest to p, and -1 or +1 is returned depending on whether point p is inside or outside the surface. The mesh is refined based on the SDF value, and the mesh quality and detail representation are optimized by inserting or deleting vertices. If |SDF(p)|>θ, point p needs to be moved to better fit the surface. The continuity of the mesh and the smoothness of the surface are optimized by resampling and adjusting the local mesh.

Images