A method for building three-dimensional reconstruction based on multi-source data alignment

Abstract

This invention relates to the field of 3D building reconstruction, specifically to a method for 3D building reconstruction based on multi-source data alignment, comprising the following steps executed sequentially: S1: Acquire 3D building feature data captured by multiple drones, and perform multi-source collaborative alignment processing on the multiple 3D building feature data to obtain aligned 3D building feature data; S2: Use an octree algorithm to recursively spatially segment the aligned 3D building feature data to obtain multiple segmentation regions; S3: Use tile-based spatial decomposition to define each segmentation region as a local center point and generate tile units; S4: Use a moving cube algorithm to extract feature points with an SDF value of 0 from the global symbolic distance field, and then construct the continuous geometric surface of the building through linear interpolation. Scale the model to the true scale using the distance reference provided by the drone's RTK-GNSS system, and output the reconstructed model; This method overcomes the limitation of traditional 3D reconstruction models that only have visual rendering effects.

Description

Technical Field

[0001] This invention relates to the field of architectural 3D reconstruction, specifically to a method for architectural 3D reconstruction based on multi-source data alignment. Background Technology

[0002] 3D architectural reconstruction refers to the process of transforming physical building entities into computer-processable digital 3D models using remote sensing, computer vision, and graphics technologies. Its core lies in acquiring the geometric structure, surface texture, and semantic information of buildings using sensor data such as images, depth, or ground data. It is commonly used in architectural scenarios with large scales and complex optical characteristics, such as the preservation of ancient buildings and engineering monitoring.

[0003] Traditional 3D architectural reconstruction relies on multi-view geometry techniques. It generates high-density point clouds through dense matching based on known poses, and then uses methods such as Poisson reconstruction to extract the 3D structure of the building. Because traditional 3D reconstruction methods are highly dependent on feature point matching in the image, they often exhibit structural distortion or voids for common architectural components such as glass, mirrors, steel structures, and solid-color curtain walls, where feature points cannot be stably extracted. Furthermore, when processing slender structures, the edges often become rounded and blurred due to matching noise and smoothing algorithms, making it impossible to reconstruct sharp architectural edges.

[0004] In recent years, with the deep integration of unmanned aerial vehicle (UAV) technology and computer vision, 3D building reconstruction technology based on Neural Radiance Fields (NeRF) has gradually replaced traditional methods, providing a new solution for digital building modeling, thanks to its advantages of continuous representation and sub-pixel accuracy. However, when performing high-precision 3D reconstruction of buildings with complex lighting conditions using multiple UAVs, existing NeRF technology has the following drawbacks:

[0005] 1. Difficulty in multi-source data fusion: In actual work, due to the differences in shooting time, lighting conditions and sensor parameters of different drones, there is a serious inconsistency in the light intensity of the overlapping observation area, which manifests as ghosting or artifacts during the reconstruction process.

[0006] 2. Challenges in large-scale building modeling: Traditional neural radiation field 3D reconstruction methods use a single neural network to represent the implicit volume density field model of a building's 3D model. When faced with kilometer-scale urban building scenarios, a single network not only struggles to fit massive amounts of spatial detail, but also experiences exponentially increasing training memory requirements and extremely slow convergence speed, failing to meet the demands of rapid industrial delivery.

[0007] 3. Lack of physical measurement capability in geometric representation: The 3D reconstruction model generated by neural radiation field technology is a continuous surface structure represented by an implicit volume density field, which contains noise and lacks clear geometric boundaries. This makes it difficult to convert the reconstructed 3D model into quantitative 3D data with physical scale (area, volume, length, etc.), and thus cannot be directly applied to subsequent building construction operations. Summary of the Invention

[0008] The purpose of this invention is to provide a method for building 3D reconstruction based on multi-source data alignment to improve surveying accuracy.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] A method for 3D building reconstruction based on multi-source data alignment includes the following steps performed sequentially:

[0011] S1: Acquire 3D building feature data from multiple drone images, and perform multi-source collaborative alignment processing on the multiple 3D building feature data to obtain aligned 3D building feature data.

[0012] S2: Use the octree algorithm to recursively spatially segment the aligned 3D building feature data to obtain multiple segmentation regions.

[0013] S3: The tile-based spatial decomposition method defines each segmented region as a local center point and generates tile elements. The specific steps for generating tile elements are as follows:

[0014] S3-1: Deploy a neural network for each tile unit to represent the local symbolic distance field of each tile unit. The local symbolic distance fields together form the global symbolic distance field. The neural network includes a reflection multilayer perceptron and a surface multilayer perceptron. The tile units are classified according to the correlation between the training light and the viewing angle. The reflection multilayer perceptron and the surface multilayer perceptron are respectively input to learn the SDF value and color information within the current tile unit. The tile units that are related to the viewing angle output the viewing angle-dependent color and volume density information through the reflection multilayer perceptron, while the tile units that are not related to the viewing angle output the viewing angle-independent color and volume density through the surface multilayer perceptron. At the same time, in order to eliminate tile boundary artifacts, adjacent tile units share the weights of the reflection multilayer perceptron.

[0015] S3-2: Using a weighting function to control the contribution of adjacent tile units This ensures a smooth transition at the boundaries of each tile unit, as shown in formula (4):

[0016] (4);

[0017] in, The distance from the sampling point to the boundary. As a smoothing factor, Indicates the boundary of the tile;

[0018] The color and density of the current sampling point are calculated by weighting and superimposing the predicted values ​​of each tile unit. The core fusion formula is shown in (5):

[0019] (5);

[0020] in, Indicates the first i The bulk density of a tile, Indicates the first i The color of each tile, The contribution of each tile is represented. Finally, the blended color and density are input into the rendering equation to render the 3D reconstructed model. The rendering equation is shown in formula (6):

[0021] (6);

[0022] Among them, transmittance This reflects the cumulative occlusion effect of all substances along the path from the sampling point to the camera. This indicates how the three-dimensional spatial coordinates change with time t. Indicates the direction of observation;

[0023] S4: Use the moving cube algorithm to extract feature points with an SDF value of 0 from the global symbolic distance field, then construct the continuous geometric surface of the building through linear interpolation, scale the model to the true scale using the distance reference provided by the UAV's RTK-GNSS system, and output the reconstructed model.

[0024] Preferably, the specific multi-source collaborative alignment processing steps in step S1 are as follows:

[0025] S1-1: Using the pose information provided by the UAV's onboard RTK-GNSS system, the building image data captured by multiple UAVs are converted to the same world coordinate system, thereby establishing a coarse alignment relationship between the multi-source data;

[0026] The specific steps are as follows: For architectural images pixels in Its relationship with three-dimensional space points Following the projective linear model, as shown in Equation (1):

[0027] (1);

[0028] Among them, scale factor This indicates the depth information of the drone within the coordinate system. The camera extrinsic matrix and camera intrinsic matrix are obtained through an airborne RTK-GNSS system. The geometric relationship between the camera's optical center and the imaging plane is defined;

[0029] S1-2: Feature matching of overlapping regions between images is performed by combining visual features and semantic information through a joint feature cost function. As shown in formula (2):

[0030] (2);

[0031] in, Represents the pixels in two images. This represents the pixel feature vector extracted using the feature extraction operator. These are the pixel category labels obtained through a semantic segmentation network. These are weighting coefficients used to adjust the contribution ratio between texture features and semantic constraints;

[0032] S1-3: Utilizing distance perception factors Adjusting reprojection error, reprojection error As shown in formula (3):

[0033] (3);

[0034] in, Indicates the first Pixel locations extracted from the image through feature matching , Indicates pixel position Position in a three-dimensional world coordinate system Represents the Huber loss function, projection operator Through the drone's intrinsic parameter matrix and extrinsic parameter matrix Calculate spatial points In the Theoretical pixel location in the image, distance perceptual factor Dynamically adjusts according to shooting distance. This represents the shooting distance of the i-th image. Indicates the minimum shooting distance. Represents the probability distribution parameters;

[0035] S1-4: Establish photometric bias mapping between images under the implicit ambient light mapping field;

[0036] S1-5: Evaluate the observation quality through back projection residuals, automatically clean up dynamic occlusion and strong reflection interference, calculate the variance of spatial points under all observation views, and if the variance exceeds the threshold, the point is determined to be a dynamic interference or instantaneous strong reflection point and is removed from the optimization weights.

[0037] Preferably, the specific segmentation steps of step S2 are as follows:

[0038] S2-1: Calculate the maximum and minimum values ​​of the aligned 3D building feature data in the X, Y, and Z directions, and construct the minimum axis-aligned bounding box that can contain all the data using the maximum and minimum values.

[0039] S2-2: Set recursion termination conditions: preset minimum data size of tiles, minimum tile size and maximum depth;

[0040] S2-3: Recursive spatial partitioning starts from the minimum axis directed bounding box: find the center point of the current cube, divide the current space into 8 sub-cubes through three mutually perpendicular planes, and distribute the feature data in the parent node to the 8 child nodes;

[0041] S2-4: Node Update: If the child node is empty, delete it directly. If the child node meets the termination condition, mark it as a leaf node and store the building data index in the area. If the termination condition is not met, use the child node as the new parent node and return to step S2-3 to continue recursion.

[0042] By adopting the aforementioned design scheme, the beneficial effects of the present invention are: the present application performs multi-source collaborative alignment processing on the acquired multiple architectural three-dimensional feature data, which effectively solves the problems of coordinate system deviation, light and shadow fluctuation and color difference of multi-source architectural image data;

[0043] By decomposing the space into tiles, the large-scale building reconstruction task is decoupled into multiple independent local sub-tasks, so that the reconstruction time no longer increases exponentially with the scene area, and the memory load requirements during the reconstruction process are significantly optimized.

[0044] By organically combining the symbolic distance field with the reflection multilayer perceptron, the limitations of traditional 3D reconstruction models that only have visual rendering effects are overcome, while avoiding voids or geometric depressions in weak texture areas such as glass curtain walls in architectural reconstruction models. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the multi-source UAV data collaborative alignment process of the present invention.

[0046] Figure 2 This is a flowchart of the 3D building reconstruction process based on tile decomposition and neural symbolic distance field of the present invention;

[0047] Figure 3 This is a flowchart of the octree space decomposition of the present invention;

[0048] Figure 4 This is a diagram illustrating the collaborative structure of the tile unit surface multilayer sensor and the reflection multilayer sensor of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0050] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0051] A method for 3D building reconstruction based on multi-source data alignment includes the following steps performed sequentially:

[0052] S1: Acquire 3D building feature data from multiple drone images, and perform multi-source collaborative alignment processing on the multiple 3D building feature data to obtain aligned 3D building feature data, such as... Figure 1 As shown, the specific steps for multi-source collaborative alignment are as follows:

[0053] S1-1: Utilizing the pose information provided by the UAV's onboard RTK-GNSS system, building image data from multiple UAVs are converted to the same world coordinate system, thereby establishing a coarse alignment relationship between the multi-source data; the specific operation is as follows: For building images... pixels in Its relationship with three-dimensional space points Following the projection linear model, in this embodiment, the coarse alignment relationship is established using the following formula (1):

[0054] (1);

[0055] Among them, scale factor This indicates the depth information of the drone within the coordinate system. The camera extrinsic matrix and camera intrinsic matrix are obtained through an airborne RTK-GNSS system. The geometric relationship between the camera's optical center and the imaging plane is defined.

[0056] S1-2: Feature matching of overlapping regions between images is performed using a joint feature cost function that combines visual and semantic information. Joint Feature Cost Function As shown in formula (2):

[0057] (2);

[0058] in, Represents the pixels in two images. This represents the pixel feature vector extracted using the feature extraction operator. These are the pixel category labels obtained through a semantic segmentation network. These are weighting coefficients used to adjust the contribution ratio between texture features and semantic constraints.

[0059] S1-3: Utilizing distance perception factors Adjusting the reprojection error resolves the resolution inconsistency between different images caused by shooting distance. Reprojection error As shown in formula (3):

[0060] (3);

[0061] in, Indicates the first Pixel locations extracted from the image through feature matching , Indicates pixel position Position in a three-dimensional world coordinate system This represents the Huber loss function. Projection operator. Through the drone's intrinsic parameter matrix and extrinsic parameter matrix Calculate spatial points In the Theoretical pixel locations in an image. Distance-aware factor. Dynamically adjusts according to shooting distance. This represents the shooting distance of the i-th image. Indicates the minimum shooting distance. Representing the probability distribution parameter, the distance perception factor forces the system to use the near-field image as the reference, preventing the blurred distant image from lowering the overall accuracy.

[0062] S1-4: Under the implicit ambient light mapping field, establish photometric bias mapping between images to achieve consistent lighting in architectural images acquired by the UAV. Perform affine transformation on the images using implicit photometric vectors to eliminate color and brightness variations caused by illumination.

[0063] S1-5: Evaluate observation quality through backprojection residuals and automatically remove dynamic occlusion and strong reflection interference. Calculate the variance of a spatial point across all observation views. If the variance exceeds a threshold, the point is identified as dynamic interference or a point with instantaneous strong reflection, and it is removed from the optimization weights.

[0064] S2: Use the octree algorithm to recursively spatially partition the aligned 3D building feature data, such as... Figure 3 As shown, the specific segmentation steps are as follows:

[0065] S2-1: Calculate the maximum and minimum values ​​of the aligned 3D building feature data in the X, Y, and Z directions, and construct the minimum axis-aligned bounding box that can contain all the data using the maximum and minimum values.

[0066] S2-2: Set recursion termination conditions: preset minimum data size of tiles, minimum tile size and maximum depth;

[0067] S2-3: Recursive spatial partitioning starts from the minimum axis directed bounding box: find the center point of the current cube, divide the current space into 8 sub-cubes through three mutually perpendicular planes, and distribute the feature data in the parent node to the 8 child nodes;

[0068] S2-4: Node Update: If the child node is empty, delete it directly. If the child node meets the termination condition, mark it as a leaf node and store the building data index in that area. If the termination condition is not met, use the child node as the new parent node and return to step S2-3 to continue recursion.

[0069] S3: Tile-based spatial decomposition is used to define each segmented region as a local center point and generate tile elements. The basic process for generating tile elements is as follows: Figure 2 As shown, the specific steps are as follows:

[0070] S3-1: A neural network is deployed for each tile unit to represent the local symbolic distance field of each tile unit. These local symbolic distance fields together form the global symbolic distance field. This neural network includes a reflection multilayer perceptron and a surface multilayer perceptron. Tile units are classified based on the correlation between training light and viewing angle, and the reflection and surface multilayer perceptrons are input separately to learn the SDF value (point-to-surface distance) and color information within the current tile unit. The basic principle is as follows: Figure 4As shown, view-dependent tile units output view-dependent color and volume density information through a reflective multilayer perceptron. View-independent tile units output view-independent color and volume density through a surface multilayer perceptron. Furthermore, to eliminate tile boundary artifacts, adjacent tile units share the weights of the reflective multilayer perceptron.

[0071] S3-2: Using a weighting function to control the contribution of adjacent tile units This ensures a smooth transition at the boundaries of each tile unit, as shown in formula (4):

[0072] (4);

[0073] in, The distance from the sampling point to the boundary. As a smoothing factor, Indicates the boundary of a tile.

[0074] The color and density of the current sampling point are calculated by weighting and superimposing the predicted values ​​of each tile unit. The core fusion formula is shown in (5):

[0075] (5);

[0076] in, Indicates the first i The bulk density of a tile, Indicates the first i The color of each tile, This indicates the contribution of each tile. Finally, the blended color and density are input into the rendering equation to render the 3D reconstructed model. The rendering equation is shown in formula (6):

[0077] (6);

[0078] Among them, transmittance This reflects the cumulative occlusion effect of all substances along the path from the sampling point to the camera. This indicates how the three-dimensional spatial coordinates change with time t. Indicates the direction of observation;

[0079] S4: The Moving Cubes algorithm is used to extract feature points with an SDF value of 0 from the global symbolic distance field, i.e., feature points with an SDF value of 0 are extracted from the SDF set. Then, the continuous geometric surface of the building is constructed through linear interpolation. Finally, the model is scaled to the true scale using the distance reference provided by the UAV's RTK-GNSS system, and the reconstructed model is output.

[0080] The foregoing description illustrates and describes embodiments of the present invention for three-dimensional building reconstruction. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other three-dimensional building reconstruction methods and can be modified within the scope of the inventive concept by means of the foregoing teachings or techniques or knowledge in related fields. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

[0081] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for 3D building reconstruction based on multi-source data alignment, characterized in that: The steps are as follows, performed sequentially: S1: Acquire 3D building feature data from multiple drone images, and perform multi-source collaborative alignment processing on the multiple 3D building feature data to obtain aligned 3D building feature data. S2: Use the octree algorithm to recursively spatially segment the aligned 3D building feature data to obtain multiple segmentation regions. S3: The tile-based spatial decomposition method defines each segmented region as a local center point and generates tile elements. The specific steps for generating tile elements are as follows: S3-1: Deploy a neural network for each tile unit to represent the local neural symbolic distance field of each tile unit. The local neural symbolic distance fields form the global symbolic distance field. The neural network includes a reflection multilayer perceptron and a surface multilayer perceptron. The tile units are classified according to the correlation between the training light and the viewing angle. The reflection multilayer perceptron and the surface multilayer perceptron are respectively input to learn the SDF value and color information within the current tile unit. The tile units that are related to the viewing angle output the viewing angle-dependent color and volume density information through the reflection multilayer perceptron, while the tile units that are not related to the viewing angle output the viewing angle-independent color and volume density through the surface multilayer perceptron. At the same time, in order to eliminate tile boundary artifacts, adjacent tile units share the weights of the reflection multilayer perceptron. S3-2: Using a weighting function to control the contribution of adjacent tile units This ensures a smooth transition at the boundaries of each tile unit, as shown in formula (4): (4)； in, The distance from the sampling point to the boundary. As a smoothing factor, Indicates the boundary of the tile; The color and density of the current sampling point are calculated by weighting and superimposing the predicted values ​​of each tile unit. The core fusion formula is shown in (5): (5)； in, Indicates the first i The bulk density of a tile, Indicates the first i The color of each tile, The contribution of each tile is represented. Finally, the blended color and density are input into the rendering equation to render the 3D reconstructed model. The rendering equation is shown in formula (6): (6)； Among them, transmittance This reflects the cumulative occlusion effect of all substances along the path from the sampling point to the camera. This indicates how the three-dimensional spatial coordinates change with time t. Indicates the direction of observation; S4: Use the moving cube algorithm to extract feature points with an SDF value of 0 from the global symbolic distance field, then construct the continuous geometric surface of the building through linear interpolation, scale the model to the true scale using the distance reference provided by the UAV's RTK-GNSS system, and output the reconstructed model.

2. The building 3D reconstruction method based on multi-source data alignment as described in claim 1, characterized in that: The specific multi-source collaborative alignment processing steps in step S1 are as follows: S1-1: Using the pose information provided by the UAV's onboard RTK-GNSS system, the building image data captured by multiple UAVs are converted to the same world coordinate system, thereby establishing a coarse alignment relationship between the multi-source data; The specific steps are as follows: For architectural images pixels in Its relationship with three-dimensional space points Following the projective linear model, as shown in Equation (1): (1)； Among them, scale factor This indicates the depth information of the drone within the coordinate system. The camera extrinsic matrix and camera intrinsic matrix are obtained through an airborne RTK-GNSS system. The geometric relationship between the camera's optical center and the imaging plane is defined; S1-2: Feature matching of overlapping regions between images is performed by combining visual features and semantic information through a joint feature cost function. As shown in formula (2): (2)； in, Represents the pixels in two images. This represents the pixel feature vector extracted using the feature extraction operator. These are the pixel category labels obtained through a semantic segmentation network. These are weighting coefficients used to adjust the contribution ratio between texture features and semantic constraints; S1-3: Utilizing distance perception factors Adjusting reprojection error, reprojection error As shown in formula (3): (3)； in, Indicates the first Pixel locations extracted from the image through feature matching , Indicates pixel position Position in a three-dimensional world coordinate system Represents the Huber loss function and projection operator. Through the drone's intrinsic parameter matrix and extrinsic parameter matrix Calculate spatial points In the Theoretical pixel location in the image, distance perceptual factor Dynamically adjusts according to shooting distance. This represents the shooting distance of the i-th image. Indicates the minimum shooting distance. Represents the probability distribution parameters; S1-4: Establish photometric bias mapping between images under the implicit ambient light mapping field; S1-5: Evaluate the observation quality through back projection residuals, automatically clean up dynamic occlusion and strong reflection interference, calculate the variance of spatial points under all observation views, and if the variance exceeds the threshold, the point is determined to be a dynamic interference or instantaneous strong reflection point and is removed from the optimization weights.

3. The building 3D reconstruction method based on multi-source data alignment as described in claim 2, characterized in that: The specific segmentation steps in step S2 are as follows: S2-1: Calculate the maximum and minimum values ​​of the aligned 3D building feature data in the X, Y, and Z directions, and construct the minimum axis-aligned bounding box that can contain all the data using the maximum and minimum values. S2-2: Set recursion termination conditions: preset minimum data size of tiles, minimum tile size and maximum depth; S2-3: Recursive spatial partitioning starts from the minimum axis directed bounding box: find the center point of the current cube, divide the current space into 8 sub-cubes through three mutually perpendicular planes, and distribute the feature data in the parent node to the 8 child nodes; S2-4: Node Update: If the child node is empty, delete it directly. If the child node meets the termination condition, mark it as a leaf node and store the building data index in the area. If the termination condition is not met, use the child node as the new parent node and return to step S2-3 to continue recursion.

Images