A building three-dimensional reconstruction method based on 3D Gaussian sputtering guidance

Abstract

The application discloses a kind of based on 3D Gaussian sputtering guide building three-dimensional reconstruction method, comprising the following steps: S1, obtain multi-view building image data;S2, based on motion recovery structure SFM from multi-view building image carries out the sparse point cloud reconstruction of building, subsequently through the initialization of building point cloud of 3D Gaussian Splatting;S3, to the 3D Gaussian point cloud after initialization carries out multi-scale optimization representation, local geometric feature accurate capture and the optimization adjustment of surface conformance, to enhance the representation of building surface;S4, based on three-dimensional Gaussian distribution generates building face piece;S5, obtains building three-dimensional model and rendering result.The scheme can restore highly detailed surface, and realize the high-precision and high-fidelity reconstruction of building geometry and appearance in complex scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a building three-dimensional reconstruction method based on 3D Gaussian sputtering guidance. BACKGROUND

[0002] Three-dimensional (3D) reconstruction of buildings refers to scanning buildings and collecting spatial information by using sensors such as RGB cameras and depth cameras to obtain a 3D geometric model of the building structure. In recent years, achieving high-quality building 3D reconstruction under complex conditions has become a research focus in the fields of photogrammetry and computer vision. High-precision 3D modeling of building structures has important application value in many fields such as urban planning, disaster simulation and cultural heritage protection.

[0003] Since the beginning of the 21st century, the types of building reconstruction methods have been increasing. Traditional three-dimensional reconstruction methods include multi-view stereo (MVS), structured light reconstruction and image-based modeling techniques, which have achieved remarkable success in simple scenes. However, when applied to scenes containing complex structures (such as buildings), these methods often face many challenges. Such scenes usually include a large amount of vegetation and roads, have complex structures, significant lighting changes and diverse perspectives, which bring great difficulties to traditional three-dimensional reconstruction methods.

[0004] With the rapid development of neural radiance field (NeRF) technology, traditional 3D reconstruction methods have made breakthrough progress. Unlike traditional explicit reconstruction techniques, NeRF generates a continuous three-dimensional scene representation by optimizing a set of directional images, thereby simulating the free expression of three-dimensional space at the image level. NeRF can generate scene reconstruction effects of imaging quality, overcoming the visual limitations of traditional 3D reconstruction methods. Although NeRF is based on a self-supervised learning model, its training process usually takes several hours, which limits its practicality.

[0005] The emergence of 3D Gaussian Splatting (3D GS) brings new hope and revolution for high-precision reconstruction of buildings in complex scenes. The 3D GS method uses Gaussian functions for scene representation, enhances adaptability in complex scene description, and shortens the training time from several hours to a few minutes. The main advantages of this method compared to traditional techniques and NeRF are: 1. The Gaussian representation of 3D GS is naturally suitable for complex scenes, enabling real-time and realistic rendering of real-world scenes. 2. It has extremely fast training speed, and preliminary rendering results can be obtained in about five minutes. 3. Unlike the implicit representation of NeRF, the 3D GS method uses explicit representation, which can directly retrain and optimize the initialized Gaussian points. However, although 3D GS can achieve realistic rendering of new perspectives in complex scenes, the distribution of Gaussian points may not fully conform to the building surface, and cannot meet the precise measurement requirements of detailed surfaces. SUMMARY

[0006] The present application aims to overcome the shortcomings of the prior art and provide a new method for complex building reconstruction based on images and guided by 3D Gaussian scattering, to restore highly detailed surfaces and achieve high-precision and high-fidelity reconstruction of building geometry and appearance in complex scenes.

[0007] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows: a 3D Gaussian scattering guided building three-dimensional reconstruction method, comprising the following steps:

[0008] S1, obtaining multi-view building image data;

[0009] S2, based on the structure from motion SFM, reconstructing the sparse point cloud of the building from the multi-view building images, and then initializing the building point cloud by 3D Gaussian Splatting;

[0010] S3, multi-scale optimization representation, accurate capture of local geometric features, and optimization adjustment of surface fitting are performed on the initialized 3D Gaussian point cloud, thereby enhancing the representation of the building surface;

[0011] S4, generating building patches based on three-dimensional Gaussian distribution;

[0012] S5, obtaining the three-dimensional model and rendering result of the building.

[0013] In step S1, multi-angle image acquisition of the building is performed by a drone carrying a camera to obtain multi-view building image data.

[0014] Step S2 includes: the structure from motion SFM first extracts feature points from the building images and matches the feature points in different view images;

[0015] When the feature point matching is completed, the three-dimensional coordinates of the matched feature points are calculated by triangulation using the corresponding positions and poses of each image, and then the camera parameters and three-dimensional point positions are globally optimized by minimizing the re-projection error, so as to obtain the building sparse point cloud and camera pose which can be used as the input of the model.

[0016] The processing of the obtained sparse point cloud based on 3D Gaussian Splatting includes: 3D Gaussian Splatting converts the sparse three-dimensional point cloud into a continuous, Gaussian-distributed state, and places a three-dimensional Gaussian distribution at the position of each point cloud data point, thereby realizing smoothing and completion of the sparse point cloud and forming an initial three-dimensional Gaussian point cloud model.

[0017] When the sparse point cloud is converted into the initial 3D Gaussian point cloud, the distribution of the initial Gaussian point cloud is represented according to the following formula:

[0018]

[0019] wherein N(x; u i ,∑ i ) represents the Gaussian distribution at the position x, u i is the mean vector of the Gaussian distribution,∑ i is the covariance matrix, the form of the Gaussian distribution ensures the smooth transition and continuity of the point cloud in the three-dimensional space,∑ i controls the shape and extension range of the Gaussian distribution, and the shape of the Gaussian point cloud is changed to adapt to different point cloud densities and distribution conditions.

[0020] In step S3, the optimization of the building point cloud by three-dimensional Gaussian point cloud optimization includes:

[0021] (1) Multi-scale Gaussian representation and fusion

[0022] The Gaussian parameters {μ s ,∑ s} of each scale are optimized for specific detail levels, and the multi-scale fusion strategy is described as the following weighted average process: the weight w s of each scale s is defined, the weight is allocated according to the importance or contribution of the scale, and the Gaussian parameters of different scales are fused to form the final scene representation:

[0023] G final =Σ s w s ·G s

[0024] wherein G s ={μ s ,Σ sis the Gaussian parameter set of the s-th scale;

[0025] (2) Regularization term based on local curvature optimization

[0026] In order to refine the Gaussian distributed point cloud in the process of building three-dimensional reconstruction, and make the Gaussian points better adapt to the scene surface geometry, a regularization term based on local curvature optimization is introduced. The regularization term focuses on punishing unnecessary curvature or noise by constraining the curvature change, so as to obtain a smoother and more accurate model. The regularization objective function is defined as:

[0027] E reg =∑ i (||∑ i -∑ target (κ i )|| 2 )

[0028] Wherein, Σ tar get (k i ) represents the target covariance matrix based on local curvature, and the local curvature κ can be calculated by the following method:

[0029]

[0030] Wherein, n is the normal vector of the point cloud.

[0031] (3) Three-dimensional Gaussian retraining for building surface fitting

[0032] Three-dimensional Gaussian retraining is performed on the building surface. In the retraining process, first, the signed distance function SDF is used to fit the building surface. SDF can accurately describe the distance and direction of points to the surface. By optimizing SDF, a smooth and accurate surface representation is obtained, providing a basis for subsequent steps. On this fitted surface, the optimization of the density function is introduced. By adjusting the density function to increase the overlap of three-dimensional Gaussians, it is ensured that the distribution of different Gaussians on the building surface is more uniform and reasonable, thereby improving the effect of three-dimensional Gaussian retraining and making it better fit the characteristics of the building surface. Finally, by reducing the scaling factor of the three-dimensional Gaussian, the three-dimensional Gaussian ellipsoid is flattened to adjust the shape of the Gaussian ellipsoid, making it more flat and fitting the surface structure of the building. By reducing the scaling factor, the accuracy and effect of three-dimensional Gaussian retraining are improved.

[0033] The building face sheet generation method based on three-dimensional Gaussian distribution in step S4 includes a building surface grid initial reconstruction step and a three-dimensional Gaussian grid joint optimization.

[0034] The building surface mesh initial reconstruction step includes obtaining a preliminary point cloud for capturing the complex geometry and details of the building after completing the 3D Gaussian point optimization of the building; resampling from the optimized Gaussian points to ensure uniform and reasonable distribution of the point cloud data, thereby better representing the surface characteristics of the object; the resampling of the optimized Gaussian points considers the change of the point cloud density function ρ(x), and adaptively adjusts the sampling rate λ(x):

[0035]

[0036] where λ min and λ max are the minimum and maximum sampling rate thresholds, respectively, and ρ th is the density threshold; subsequently, the Poisson reconstruction method is used to generate an initial triangular mesh, and the point cloud data is smoothed by constructing a global least squares solution to generate a continuous and consistent triangular mesh surface, and the divergence operator div is used to calculate the divergence of the vector field v generated by the optimized Gaussian points, which represents the direction and intensity distribution of the point cloud, and is a key to describe the surface characteristics of the building;

[0037] Then the Laplacian operator Δ is used to calculate the second derivative of the function f, and the Poisson equation is solved:

[0038] Δf=div v

[0039] to obtain a continuous function f consistent with the point cloud data, and the solution constitutes the initial mesh surface of the building.

[0040] The three-dimensional Gaussian mesh joint optimization step includes: using SDF to refine and optimize the three-dimensional mesh, SDF defines a distance value for each spatial point, which represents the distance from the point to the nearest surface, where a positive value indicates that the point is outside the object, and a negative value indicates that the point is inside the object; for each mesh point p, calculate its SDF value SDF(p):

[0041] SDF(p)=sign(p)·min q∈Surface ||p-q||

[0042] where sign(p) is the sign function, p is a point in the mesh, and q is the nearest surface point from p, which returns -1 or +1 according to whether the point p is inside or outside the surface; based on the SDF value, the mesh is refined, and the mesh quality and detail representation are optimized by inserting or deleting vertices; if |SDF(p)|> θ, the point p needs to be moved to better adapt to the surface; the continuity and smoothness of the mesh are optimized by local mesh resampling and adjustment.

[0043] The advantages of the present application are that: 1. 3D Gaussian scattering is used to guide the smoothing and completion of the sparse point cloud generated by SFM, creating a high-density point cloud, significantly enhancing the continuity and realism of the point cloud, thereby laying a solid foundation for the subsequent geometric reconstruction of the building.

[0044] 2. A three-dimensional Gaussian point cloud optimization strategy is designed for complex building surface structures, multi-scale Gaussian representation and fusion technology are used, and a local curvature optimization regularization term is combined, so that the 3D Gaussian points more accurately conform to the building surface geometry, enhance the ability to capture building surface details, and improve the smoothness and visual consistency of the reconstructed model.

[0045] 3. Poisson algorithm is used to optimize the surface of complex buildings to reconstruct the initial triangular mesh, and Signed Distance Fields (SDF) are used to further optimize the geometric accuracy and surface detail representation, thereby improving the detail fidelity and overall reconstruction quality of complex building surfaces. BRIEF DESCRIPTION OF DRAWINGS

[0046] The content expressed by each figure in the specification of the present application and the marks in the figures are briefly described as follows:

[0047] Figure 1 The SFM sparse point cloud generation flowchart of the present application;

[0048] Figure 2 The point cloud generated by the present application; wherein (a) is the sparse point cloud generated by SFM; (b) is the point cloud initialized by 3D Gaussian. DETAILED DESCRIPTION

[0049] The specific embodiments of the present application are further described in detail below by comparing the figures and describing the optimal embodiments.

[0050] The present application proposes a new method for complex building reconstruction based on images and guided by 3D Gaussian scattering to restore highly detailed surfaces and achieve high-precision and high-fidelity reconstruction of building geometry and appearance in complex scenes. The main contributions of the present application include:

[0051] 1. 3D Gaussian scattering is used to guide the smoothing and completion of the sparse point cloud generated by SFM, creating a high-density point cloud, significantly enhancing the continuity and realism of the point cloud, thereby laying a solid foundation for the subsequent geometric reconstruction of the building.

[0052] 2. A three-dimensional Gaussian point cloud optimization strategy is designed for complex building surface structures, multi-scale Gaussian representation and fusion technology are used, and a local curvature optimization regularization term is combined, so that the 3D Gaussian points more accurately conform to the building surface geometry, enhance the ability to capture building surface details, and improve the smoothness and visual consistency of the reconstructed model.

[0053] 3. Use Poisson algorithm to optimize the surface of complex buildings to reconstruct the initial triangular mesh, and use Signed Distance Fields (SDF) to further optimize the geometric precision and surface detail representation, so as to improve the detail fidelity and overall reconstruction quality of the surface of complex buildings.

[0054] As shown in Figure 1 , 2 The technical scheme adopted by the present application is as follows: a new method for reconstructing complex buildings based on images and guided by 3D Gaussian scattering includes the following steps:

[0055] S1, obtaining a multi-view building image dataset;

[0056] S2, three-dimensional Gaussian point cloud initialization: based on structure from motion (SFM), sparse point cloud reconstruction of the building is performed from the multi-view building image, and then the building point cloud is initialized through 3D Gaussian splatting;

[0057] S3, three-dimensional Gaussian point cloud optimization suitable for buildings: multi-scale optimization representation, accurate capture of local geometric features, and optimization adjustment of surface fitting are performed on the initialized 3D Gaussian point cloud, so as to enhance the representation of the building surface.

[0058] S4, building patch generation method based on three-dimensional Gaussian distribution;

[0059] S5, obtaining building three-dimensional model and rendering result; patch is the core part of constructing three-dimensional model, but simply generating patch is not enough to form complete three-dimensional model, patch needs to be connected according to topological rules to form closed or continuous surface structure (building three-dimensional model). The building three-dimensional model is generated by poisson algorithm from the optimized Gaussian point cloud to extract the patch and generate the complete three-dimensional model. The patch itself does not contain visual information such as texture and lighting, and needs to obtain the rendering result through texture mapping. Among them:

[0060] For texture extraction: extract texture information from multi-view images and calculate the corresponding UV texture coordinates of each patch.

[0061] Map to patch: map the texture image to the patch to provide color and material information for the model.

[0062] In step S1, the multi-view building image dataset is composed of a Small Buildings dataset provided by ArcGIS company and a Tower dataset collected by a UAV; when the algorithm is verified, a public dataset and a self-measured dataset can be used, and when the building reconstruction is performed using the method, images of multiple angles of the building to be reconstructed need to be collected by a UAV or other means.

[0063] In step S2, the SFM algorithm first extracts feature points from the building images and matches the feature points in different view images. When the feature point matching is completed, the three-dimensional coordinates of the matched feature points are calculated by triangulation using the corresponding positions and poses of each image, and then the camera parameters and the three-dimensional point positions are globally optimized by minimizing the re-projection error, so as to obtain a building sparse point cloud and camera poses which can be used as model input. Subsequently, the obtained sparse point cloud is processed based on 3D Gaussian Splatting. The 3D Gaussian Splatting can convert the sparse three-dimensional point cloud into a continuous, Gaussian-distributed state. The basic principle is to place a three-dimensional Gaussian distribution at the position of each point cloud data point, so as to realize the smoothing and completion of the sparse point cloud and form an initial 3D Gaussian point cloud.

[0064] The optimization of parameters and positions by minimizing the re-projection error includes: the sparse point cloud and the camera pose result directly affect the accuracy of subsequent Gaussian reconstruction. The sparse point cloud and the camera pose are optimized by minimizing the error function G(C, X), and the model accuracy is evaluated.

[0065]

[0066] where ρ ij represents an indicator function; q ij represents an observed image point; C i and X j represent camera parameters and three-dimensional point coordinates, respectively; P(C i , X j ) represents a projection point of a three-dimensional point in the camera view. By minimizing the error function, the SFM algorithm refines the camera parameters and the three-dimensional point positions to produce an accurate sparse point cloud representation.

[0067] In the method, the sparse point cloud is converted into an initial 3D Gaussian point cloud, and the distribution of the initial Gaussian point cloud is represented according to the following formula:

[0068]

[0069] where N(x; u i , Σ i ) represents a Gaussian distribution at position x, ui is the mean vector of the Gaussian distribution, Σ i is the covariance matrix, the form of Gaussian distribution ensures the smooth transition and continuity of point cloud in three-dimensional space, Σ i controls the shape and range of expansion of the Gaussian distribution. The shape of the Gaussian point cloud can be changed to adapt to different point cloud density and distribution. The letter T represents the transpose of the vector, which is a mathematical operator, (x-u i ) T is the transpose of the column vector (x-u i ), which is changed into a row vector.

[0070] In step S3, the three-dimensional Gaussian point cloud suitable for buildings is optimized, wherein the optimization scheme includes:

[0071] (1) Multi-scale Gaussian representation and fusion

[0072] When dealing with complex objects such as buildings, multi-scale Gaussian representation can better capture the macro and micro details of the building, and optimize the Gaussian parameters {μ s ,∑ s} for specific levels of detail at each scale. The strategy of multi-scale fusion can be mathematically described as the following weighted average process: define the weight w s of each scale s, these weights are allocated according to the importance or contribution of the scale, and the Gaussian parameters of different scales are fused to form the final scene representation. As shown in the following formula:

[0073] G final =∑ s w s ·G s

[0074] where G s ={μ s ,∑ s} is the Gaussian parameter set of the s-th scale. This process ensures the richness of details and the naturalness of visual effects in the reconstruction process, and can effectively balance the preservation of details and the overall consistency of the scene.

[0075] (2) Regularization term based on local curvature optimization

[0076] In order to further refine the Gaussian distribution point cloud in the three-dimensional reconstruction process of buildings, so that the Gaussian points better adapt to the scene surface geometry, a regularization term based on local curvature optimization is introduced, which focuses on punishing unnecessary curvature or noise by constraining the curvature change, so as to obtain a smoother and more accurate model. The regularization objective function is defined as:

[0077] E reg =∑ i (||∑i -∑ target (κ i )|| 2

[0078] where, ∑ target (κ i ) represents the target covariance matrix based on local curvature, and the local curvature κ can be calculated by the following method:

[0079]

[0080] Here, n is the normal vector of the point cloud. The local curvature reflects the deformation characteristics of the point cloud in a small range, effectively reducing errors caused by noise and data inconsistency, thereby improving the geometric accuracy and visual effect of the reconstructed scene to a greater extent. This adjustment can enhance the details and realism of the three-dimensional reconstruction, and to some extent, it can improve the ability of the method to handle complex architectural features, ensuring high-quality reconstruction results

[0081] The operation required to introduce the regularization term is:

[0082] (1) Calculate the local curvature of the point cloud

[0083]

[0084] (2) Define the regularization term

[0085] Use the value of the local curvature as the core parameter of the regularization constraint to construct the target covariance matrix to limit the local geometric shape of the points and ensure the smoothness of the transition.

[0086] E reg =∑ i (||∑ i -∑ target (κ i )||2 2 )(3) Three-dimensional Gaussian retraining for building surface fitting

[0087] Although the improved 3D GS method can obtain a high-fidelity three-dimensional building effect after fine-tuning the distribution of points, there is currently no method that can directly measure from Gaussian points. The method of directly based on Gaussian points cannot generate triangular meshes that fit the building surface when it is faceted, so three-dimensional Gaussian retraining is needed for the building surface. Steps (1) and (2) in step S3 are the synchronous optimization of the point cloud, and step (3) is the retraining of the optimized point cloud.

[0088] ​In the retraining process, the signed distance function (SDF) is first used to fit the building surface. The SDF can accurately describe the distance and direction from a point to the surface. By optimizing the SDF, a smooth and accurate surface representation can be obtained, thus providing a foundation for subsequent steps. On this fitted surface, the density function is optimized. By adjusting the density function, the overlap of the three-dimensional Gaussians is increased, ensuring that the distribution of different Gaussians on the building surface is more uniform and reasonable, thereby improving the effect of the three-dimensional Gaussian retraining and making it better fit the features of the building surface. Finally, the three-dimensional Gaussian ellipsoid is flattened by reducing the scaling factor of the three-dimensional Gaussian to adjust the shape of the Gaussian ellipsoid, making it flatter and more in line with the surface structure of the building. By reducing the scaling factor, the accuracy and effect of the three-dimensional Gaussian retraining can be further improved.

[0089] During the retraining process, the signed distance function (SDF) is first used to fit the building surface. The SDF can accurately describe the distance from a point to the surface and its direction. The formula is as follows:

[0090]

[0091] Here, d(x, S) represents the Euclidean distance from point x to surface S. By optimizing the SDF, a smooth and accurate surface representation can be obtained, thus providing a foundation for subsequent steps.

[0092] On this fitted surface, density function optimization is introduced. By adjusting the density function ρ(x), the overlap of the three-dimensional Gaussians is increased, ensuring a more uniform and reasonable distribution of different Gaussians on the building surface. The density function optimization can be expressed as:

[0093]

[0094] Where, α i It is the weight of the i-th Gaussian point, x i This represents the location of the Gaussian point, where σ is the standard deviation of the Gaussian function, and is determined by adjusting α. i σ can control the density distribution, thereby optimizing the overlap and improving the effect of 3D Gaussian retraining, making it better fit the surface features of buildings.

[0095] Finally, the 3D Gaussian ellipsoid is flattened by reducing the 3D Gaussian scaling factor to adjust its shape, making it flatter and more closely conforming to the building's surface structure. The shape of the Gaussian ellipsoid is typically described by the scaling factor s and the principal axis lengths λ1, λ2, and λ3, as shown in the formula:

[0096]

[0097] where x = (x1, x2, x3) is an arbitrary point in three-dimensional space, and s can be adjusted to make the Gaussian ellipsoid more flat, thus better fitting the building surface. By optimizing the scaling factor, the accuracy and effectiveness of the three-dimensional Gaussian retraining are further improved.

[0098] In step S4, the building face sheet generation method based on three-dimensional Gaussian distribution specifically includes:

[0099] (1) Building surface grid initial reconstruction

[0100] After completing the 3D Gaussian point cloud optimization of the building, a preliminary point cloud that can be used to capture the complex geometry and details of the building is obtained. Next, resampling is performed from the optimized Gaussian point cloud to ensure uniform and reasonable distribution of point cloud data, so as to better represent the surface features of the object, which is crucial for generating accurate triangular meshes. The resampling of the optimized Gaussian point cloud considers the change of the point cloud density function ρ(x), and adaptively adjusts the sampling rate λ(x):

[0101]

[0102] where λ min and λ max are the minimum and maximum sampling rate thresholds, respectively, and ρ th is the density threshold. Through the adaptive sampling strategy, the number of sampling points in high-density areas is effectively reduced, avoiding over-sampling, and the number of sampling points in low-density areas is increased to capture finer details.

[0103] For the point cloud data obtained after resampling, we use the Poisson reconstruction method to generate the initial triangular mesh. This method smoothes the point cloud data by constructing a global least squares solution, thereby generating a continuous and consistent triangular mesh surface. The divergence operator div is used to calculate the divergence of the vector field v generated by the optimized Gaussian points, which represents the direction and intensity distribution of the point cloud and is the key to describing the features of the building surface.

[0104] Then the Laplacian operator Δ is used to calculate the second derivative of the function f, by solving the Poisson equation:

[0105] Δf = div v

[0106] We can get a continuous function f consistent with the point cloud data, whose solution constitutes the initial mesh surface of the building. This method ensures the smoothness and consistency of the mesh, while maintaining the accurate expression of geometric details.

[0107] Poisson reconstruction method is known for its robustness and efficiency, which can effectively handle complex surface shapes and generate high-quality triangular meshes. Compared with other reconstruction methods, Poisson reconstruction performs well in handling noise and non-uniform sampling point clouds, and can better preserve the details and complex structures of objects.

[0108] (2) Joint optimization of three-dimensional Gaussian mesh

[0109] Poisson reconstruction method is widely used to generate initial three-dimensional meshes, but it has obvious limitations. First, Poisson reconstruction tends to generate smooth surfaces, which may lead to loss of details, especially when dealing with objects with complex surface features. In addition, in areas with excessive filling or missing data, Poisson reconstruction may generate unrealistic surfaces, resulting in inaccurate edges, especially at boundaries or irregular edges, thereby affecting the overall quality of the three-dimensional model.

[0110] To solve these problems, SDF is used to refine and optimize the mesh. The SDF method enhances the ability to handle details of the original point cloud, provides better noise resistance, and has higher efficiency in handling large-scale data. Compared with Poisson reconstruction, SDF performs well in filling data gaps and generating realistic surfaces, while producing accurate edges at open or irregular boundaries, thereby significantly improving the overall quality of the three-dimensional model.

[0111] Poisson reconstruction gets the initialization three-dimensional network, and then further refines and optimizes the three-dimensional mesh through SDF. SDF is used to further refine and optimize the three-dimensional mesh. SDF defines a distance value for each spatial point, which represents the distance from the point to the nearest surface, where positive values indicate points outside the object and negative values indicate points inside the object. For each mesh point p, calculate its SDF value SDF(p):

[0112] SDF(p) = sign(p) · min q∈Surface ||p-q||

[0113] where sign(p) is a function, p is a point in the mesh, and q is the nearest surface point to p, returning -1 or +1 depending on whether the point p is inside or outside the surface. Based on the SDF value, the mesh is refined by inserting or deleting vertices to optimize mesh quality and detail representation, and if |SDF(p)|>θ, the point p needs to be moved to better adapt to the surface, where θ represents a threshold parameter for determining whether the mesh point p needs to be adjusted. Through local mesh resampling and adjustment, the continuity and smoothness of the mesh surface are optimized.

[0114] Refining the mesh based on the SDF value includes:

[0115] 1. Insert new points to optimize mesh quality

[0116] When the mesh triangles in a certain region are too large or too sparse, new points need to be inserted to refine the local triangle mesh structure, maintaining local smoothness.

[0117] 2. Move points to better adapt to the surface

[0118] When the mesh points deviate from the target surface (i.e. SDF value close to but not zero), the points need to be moved

[0119] 3. Delete points to optimize the mesh

[0120] When the point density in a certain region is too high, and the distribution of points does not significantly contribute to the surface characteristics, redundant points need to be deleted.

[0121] 4. Resampling and adjustment of local mesh

[0122] When the continuity or smoothness of the local mesh is poor (e.g. sharp curvature changes or irregular mesh topology), the mesh points need to be resampled.

[0123] Curvature adaptive sampling: adjust the point density based on local curvature, adaptively increase or decrease sampling points according to the density function.

[0124] Mesh adjustment: regenerate the local triangle mesh based on the resampled point cloud, ensuring that the mesh shape is regular and the surface is smooth.

[0125] Experimental result analysis:

[0126] The reconstruction method provided in this scheme is compared with the traditional method colmap and the latest neural radiance field method neuralangelo. As a representative of the classic method, colmap relies on multi-view geometry (SFM) and multi-view stereo matching (MVS) to recover sparse point clouds and generate dense 3D reconstruction models. Similarly, we also use colmap to recover sparse point clouds from images through SFM in the initial process as the basis for the reconstruction method of this scheme. While neuralangelo performs coarse-to-fine optimization on the hash grid at each level of detail, reconstructing high-fidelity 3D surface structures from multi-view images, achieving the most advanced results. The performance of each method in three-dimensional model generation and rendering tasks is analyzed.

[0127] To quantitatively evaluate the performance of colmap, neuralangelo and the proposed method, PSNR and SSIM are used as evaluation metrics. After the network training is completed, a certain number of images are randomly extracted from the image dataset for testing. According to the camera poses of these images, the corresponding rendering images are generated for the three-dimensional model, and the PSNR and SSIM values between the rendering images and the original images are calculated, and the average values are taken as the final overall evaluation results of each method. Table 1 compares the PSNR and SSIM values of the three methods on the rendering images, and the results show the advantages of the proposed method in terms of fidelity and structural consistency.

[0128] Table 1. Comparison of PSNR and SSIM values of three methods

[0129]

[0130] As shown in Table 1, for the comparison of PSNR and SSIM values of the two datasets, the proposed method is significantly better than colmap and neuralangelo methods in both PSNR and SSIM indicators.

[0131] On the Small Buildings dataset, it can be observed that the PSNR and SSIM values of the neuralangelo method are the lowest, 17.31 and 0.76 respectively, which indicates that the rendering effect of neuralangelo has a large difference from the pixel values of the original image, with a lot of noise and distortion, and it cannot effectively restore the details in the original image. Moreover, the rendering image has significant differences in brightness, contrast and structural information from the original image, with relatively low structural similarity and poor visual quality. Although the colmap method has improved on the basis of neuralangelo, it is still difficult to achieve ideal results. In contrast, our method has far exceeded the performance of colmap and neuralangelo, with PSNR and SSIM values of 29.67 and 0.94. This significant improvement shows that our method provides more accurate surface reconstruction, effectively reduces noise and distortion while preserving details, making the rendering image closer to the visual effect of the original image.

[0132] On the Tower dataset, colmap and neuralangelo methods have similar performance, with PSNR values of 25.18 and 24.27 respectively, and SSIM values of 0.91 and 0.87 respectively. However, our method has achieved PSNR and SSIM values of 32.69 and 0.96, which shows that our method is more effective in preserving the complex structural details and texture consistency in the original image, especially in the fine-grained representation of building structures, which is crucial for complex architectural structures.

[0133] Obviously, the specific implementation of the present application is not limited by the above-mentioned manner, as long as various non-essential improvements are made by adopting the method concept and technical solutions of the present application, which are within the protection scope of the present application.

Claims

1. A method for three-dimensional reconstruction of buildings based on 3D Gaussian sputtering guidance, characterized in that: Includes the following steps: S1. Acquire multi-view building image data; S2. Sparse point cloud reconstruction of buildings is performed from multi-view building images based on structure of motion restoration (SFM), and then the building point cloud is initialized by 3D Gaussian Splatting. S3. Perform multi-scale optimization representation, accurate capture of local geometric features, and optimization adjustment of surface fit on the initialized 3D Gaussian point cloud to enhance the representation of the building surface. S4. Generate building patches based on three-dimensional Gaussian distribution; S5. Obtain the 3D model and rendering results of the building; In step S3, the optimization process for the building point cloud using 3D Gaussian point cloud optimization includes: (1) Multiscale Gaussian Representation and Fusion The Gaussian parameter {μ} at each scale s , ∑ s For optimization at specific levels of detail, the multi-scale fusion strategy can be mathematically described as the following weight averaging process: Define the weight w for each scale s. s The weights are assigned based on the importance or contribution of each scale, and Gaussian parameters from different scales are combined to form the final scene representation. G final =∑ s w s ·G s Among them, G s ={μ s , ∑ s } is the set of Gaussian parameters at the s-th scale; (2) Based on local curvature optimization regularization term To refine the Gaussian point cloud during the 3D reconstruction of buildings and better adapt the Gaussian points to the scene surface geometry, a regularization term based on local curvature optimization is introduced. This regularization term constrains curvature variations, primarily penalizing unnecessary curvature or noise, thereby obtaining a smoother and more accurate model. The regularization objective function is defined as: E reg =∑ i (||∑ i -∑ target (k i )|| 2 ) Where, ∑ target (κ i ) represents the target covariance matrix based on local curvature, ∑ i The covariance matrix and the local curvature κ can be calculated using the following method: Where n is the normal vector of the point cloud; (3) Three-dimensional Gaussian retraining for building surface bonding A 3D Gaussian retraining process is performed on the building surface. First, the signed distance function (SDF) is used to fit the building surface. The SDF accurately describes the distance and direction from points to the surface. By optimizing the SDF, a smooth and accurate surface representation is obtained, providing a foundation for subsequent steps. Next, density function optimization is introduced on this fitted surface. Adjusting the density function increases the overlap of the 3D Gaussians, ensuring a more uniform and reasonable distribution of different Gaussians on the building surface, thereby improving the 3D Gaussian retraining effect and making it better fit the building surface features. Finally, the 3D Gaussian ellipsoid is flattened by reducing the scaling factor to adjust its shape, making it flatter and more closely fit the building's surface structure. By reducing the scaling factor, the accuracy and effect of the 3D Gaussian retraining are improved.

2. The method for three-dimensional reconstruction of buildings based on 3D Gaussian sputtering as described in claim 1, characterized in that: In step S1, a drone equipped with a camera is used to collect multi-angle images of the building to obtain multi-view building image data.

3. The method for three-dimensional reconstruction of buildings based on 3D Gaussian sputtering as described in claim 1, characterized in that: Step S2 includes: The motion recovery structure SFM first extracts feature points from the building image and matches the feature points in images from different viewpoints; Once feature point matching is complete, the three-dimensional coordinates of the matched feature points are calculated using triangulation based on the corresponding position and pose of each image. Then, by minimizing the reprojection error, the camera parameters and the three-dimensional point positions are globally optimized to obtain sparse point clouds of buildings and camera poses that can be used as model inputs.

4. The method for three-dimensional reconstruction of buildings based on 3D Gaussian sputtering as described in claim 3, characterized in that: The sparse point cloud processing based on 3D Gaussian Splatting includes: 3D Gaussian Splatting transforms the sparse 3D point cloud into a continuous, Gaussian distributed state, and places a 3D Gaussian distribution at the location of each point cloud data point, thereby achieving smoothing and completion of the sparse point cloud and forming an initial 3D Gaussian point cloud model.

5. The method for three-dimensional reconstruction of buildings based on 3D Gaussian sputtering guidance as described in claim 4, characterized in that: When transforming a sparse point cloud into an initial 3D Gaussian point cloud, the distribution of the initial Gaussian point cloud is represented by the following formula: Wherein, N(x; u) i , ∑ i ) represents a Gaussian distribution at position x, u i It is the mean vector of a Gaussian distribution. The form of the Gaussian distribution ensures the smooth transition and continuity of the point cloud in three-dimensional space. ∑ i The shape and extent of the Gaussian distribution were controlled, and the shape of the Gaussian point cloud was changed to adapt to different point cloud densities and distributions.

6. A method for three-dimensional reconstruction of buildings based on 3D Gaussian sputtering as described in any one of claims 1-5, characterized in that: The method for generating building patches based on three-dimensional Gaussian distribution in step S4 includes: initial reconstruction of the building surface mesh and joint optimization of the three-dimensional Gaussian mesh.

7. The method for three-dimensional reconstruction of buildings based on 3D Gaussian sputtering guidance as described in claim 6, characterized in that: The initial reconstruction step of the building surface mesh includes obtaining a preliminary point cloud after completing the 3D Gaussian point optimization of the building to capture the complex geometry and details of the building; resampling is performed from the optimized Gaussian points to ensure the uniform and reasonable distribution of the point cloud data, so as to better represent the surface features of the object; Optimized Gaussian point resampling takes into account the variation of the point cloud density function ρ(x) and adaptively adjusts the sampling rate λ(x): Where, λ min and λ max These are the minimum and maximum sampling rate thresholds, ρ th The density threshold is set; then, the Poisson reconstruction method is used to generate an initial triangular mesh, and the point cloud data is smoothed by constructing a global least squares solution to generate a continuous and consistent triangular mesh surface. The divergence operator div is used to calculate the divergence of the vector field v generated by the optimized Gaussian points. This vector field v represents the direction and intensity distribution of the point cloud and is the key to describing the surface features of the building. Next, the second derivative of the function f is calculated using the Laplace operator Δ, and the Poisson equation is solved: Δ=div v A continuous function f consistent with the point cloud data is obtained, and its solution constitutes the initial grid surface of the building.

8. A method for three-dimensional reconstruction of buildings based on 3D Gaussian sputtering as described in any one of claims 1-5, characterized in that: The joint optimization steps for the 3D Gaussian mesh include: using SDF to refine and optimize the 3D mesh. SDF defines a distance value for each spatial point, representing the distance from the point to the nearest surface, where a positive value indicates the point is outside the object, and a negative value indicates the point is inside the object; for each mesh point p, its SDF value SDF(p) is calculated: SDF(p)=sign(p)·min q∈Surface ||p-q|| Here, sign(p) is the sign function, p is a point in the mesh, and q is the nearest surface point to p. It returns -1 or +1 depending on whether point p is inside or outside the surface. The mesh is refined based on the SDF value, and the mesh quality and detail representation are optimized by inserting or deleting vertices. If |SDF(p)|>θ, then point p needs to be moved to better fit the surface. The continuity of the mesh and the smoothness of the surface are optimized by local mesh resampling and adjustment.

Images