Large-scale aviation city market scenario reconstruction method based on divide-and-conquer strategy

Abstract

This application relates to a large-scale aerospace city scene reconstruction method based on a divide-and-conquer strategy. The method includes: performing coordinate system unification processing on an initial point cloud to obtain a two-dimensional point cloud set; adaptively partitioning the spatial distribution density of the two-dimensional point cloud set to divide the complete scene into multiple spatial sub-blocks; partitioning training data for each spatial sub-block based on visibility perception, and assigning a corresponding training image set to each spatial sub-block; constructing a three-dimensional Gaussian splash model for each spatial sub-block, performing parallel training using the assigned training image set, and optimizing the Gaussian meta-parameters by combining a joint loss function consisting of photometric loss, depth loss, pseudo-view geometric consistency loss, and scale regularization loss; and merging all trained spatial sub-blocks to generate a three-dimensional reconstruction result of the complete scene. This method can achieve computational load balancing, ensure inter-block geometric consistency, suppress floating artifacts, and significantly improve training efficiency and reconstruction quality.

Description

Technical Field

[0001] This application relates to the field of 3D reconstruction technology, and in particular to a method for large-scale aviation city scene reconstruction based on a divide-and-conquer strategy. Background Technology

[0002] In recent years, with the popularization of the digital twin concept and the deepening of smart city construction, 3D reconstruction technology has become a research hotspot in the fields of computer vision and computer graphics. Among numerous application scenarios, the construction of large-scale urban reality models is particularly crucial. Currently, the construction of urban reality models based on aerial images mainly relies on oblique photogrammetry techniques centered on Structure from Motion (SfM) and Multi-View Stereo (MVS). Although this technology system is relatively mature, it still faces significant limitations when dealing with the needs of large-scale, high-fidelity urban reconstruction.

[0003] In recent years, new viewpoint synthesis technologies, represented by Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS), have rapidly become research hotspots in the field of 3D reconstruction. Compared to NeRF, 3DGS, with its explicit Gaussian representation and efficient differentiable rasterization technology, achieves real-time rendering speed while maintaining high-fidelity rendering. However, directly applying 3DGS to aerial photography still faces many challenges: 1) Graphical memory bottleneck: As the scene size increases, the required number of Gaussian primitives grows dramatically, easily exceeding the current graphics card memory limit; 2) Limited representation capability: If the number of Gaussian primitives is limited, sparse Gaussian primitives will not be able to cover a vast scene, resulting in texture blurring and loss of geometric details; 3) Difficulty in optimization convergence: The global optimization solution space is too large, and the preset number of iterations is far from sufficient for the model to converge; 4) Floating artifacts: In the absence of effective constraints, the model tends to generate a large number of invalid floating artifacts, destroying the geometric continuity of the scene.

[0004] To address the aforementioned issues, existing techniques such as MegaNeRF, SwitchNeRF, VastGaussian, and CityGaussian employ a divide-and-conquer strategy. However, these techniques still have the following drawbacks: First, most employ a simple uniform grid partitioning strategy, ignoring the non-uniform distribution of spatial point cloud density, leading to unbalanced computational load and limiting the efficiency of multi-GPU parallel training. Second, relying solely on COLMAP results for training data partitioning makes it difficult to guarantee geometric consistency between blocks, easily introducing geometric errors and artifacts at the seams. Third, some methods depend on expensive global pre-training, reducing overall training efficiency. Fourth, most fail to solve the problem of numerous floating artifacts present in the reconstruction results. Summary of the Invention

[0005] Therefore, it is necessary to provide a large-scale aviation city scene reconstruction method based on a divide-and-conquer strategy that can achieve computational load balancing, ensure geometric consistency between blocks, suppress floating artifacts, and improve training efficiency and reconstruction quality, in order to address the above-mentioned technical problems.

[0006] A method for reconstructing large-scale aerospace city scenes based on a divide-and-conquer strategy, the method comprising: Acquire initial point cloud and camera pose data for large-scale aerospace city scenes; perform coordinate system normalization processing on the initial point cloud to obtain a two-dimensional point cloud set; Adaptive spatial partitioning is performed on the spatial distribution density of the 2D point cloud set to divide the complete scene into multiple spatial sub-blocks; training data is partitioned for each spatial sub-block based on visibility perception, and a corresponding training image set is assigned to each spatial sub-block; A 3D Gaussian splash model is constructed for each spatial sub-block. The 3D Gaussian splash model is trained in parallel using the allocated training image set. During the training process, the Gaussian meta-parameters in each spatial sub-block are optimized by combining the joint loss function consisting of photometric loss, depth loss, pseudo-view geometric consistency loss and scale regularization loss. Merge all trained spatial sub-blocks of the 3D Gaussian splash model to generate the complete 3D reconstruction result of the scene.

[0007] The aforementioned large-scale aviation city scene reconstruction method based on a divide-and-conquer strategy first performs coordinate system unification on the initial point cloud to obtain a two-dimensional point cloud set, laying a unified spatial benchmark for subsequent adaptive spatial partitioning. Then, it adaptively partitions the spatial distribution density of the two-dimensional point cloud set, dividing the complete scene into multiple spatial sub-blocks. This achieves adaptive scene partitioning based on spatial point cloud density awareness, solving the problem of unbalanced computational load caused by uniform grid partitioning in existing technologies, effectively balancing the computational load of each sub-block, and improving the resource utilization of multi-GPU parallel training. Next, it partitions the training data for each spatial sub-block based on visibility awareness, assigning a corresponding training image set to each sub-block, and introducing block boundary dilation and auxiliary point enhancement strategies. This solves the problem of mismatch between the boundary view and the supervision image caused by non-uniform partitioning, ensuring that each sub-scene can obtain sufficient and accurate geometric supervision signals, thereby guaranteeing high-quality local reconstruction and inter-block geometric consistency. Finally, during training, a joint loss function consisting of photometric loss, depth loss, pseudoview geometric consistency loss, and scale regularization loss was combined to effectively suppress near-field floating artifacts by utilizing depth priors. At the same time, the spatial expansion of Gaussian units was constrained, avoiding the occurrence of Gaussian points with scale anomalies. This improved the visual fidelity of the model under new perspectives, achieved computational load balancing, ensured inter-block geometric consistency, suppressed floating artifacts, and significantly improved training efficiency and reconstruction quality. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating a large-scale aviation city scene reconstruction method based on a divide-and-conquer strategy in one embodiment. Figure 2 This is a schematic diagram illustrating the determination of the minimum bounding box and orientation alignment in one embodiment; Figure 3 This is a schematic diagram of the scene adaptive partitioning result in one embodiment; Figure 4 This is a flowchart illustrating a method for mismatch between the effective field of view and supervisory information in a sub-region, as described in another embodiment. Detailed Implementation

[0009] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0010] In one embodiment, such as Figure 1 As shown, a method for large-scale aerospace city scene reconstruction based on a divide-and-conquer strategy is provided, including the following steps: Step 102: Obtain the initial point cloud and camera pose data of the large-scale aviation city scene; perform coordinate system normalization processing on the initial point cloud to obtain a two-dimensional point cloud set.

[0011] The initial point cloud is a sparse point cloud reconstructed from aerial images using the Structure for Motion Restoration (SfM) algorithm. Since the coordinate system of the point cloud used for 3DGS initialization is determined by the SfM algorithm, it is random and arbitrary. This inconsistency in the coordinate system hinders the implementation of a universal partitioning strategy. By performing coordinate system unification processing on the initial point cloud, all point clouds and camera poses are transformed into a unified reference world coordinate system, solving the coordinate system inconsistency problem. This lays a unified spatial reference for subsequent adaptive spatial partitioning, enabling spatial partitioning based on point cloud density to accurately reflect the true geometric distribution of the scene.

[0012] Step 104: Adaptively divide the spatial distribution density of the two-dimensional point cloud set into multiple spatial sub-blocks; divide the training data for each spatial sub-block based on visibility perception, and assign a corresponding training image set to each spatial sub-block.

[0013] High-density regions in point clouds typically imply complex geometric details, resulting in a large number of Gaussian primitives generated during training. This leads to high memory consumption and long convergence times, necessitating a limitation on the partitioning range of such regions. Conversely, sparse regions in point clouds have lower computational loads, and overly fine partitioning can lead to insufficient GPU resource utilization. By adaptively partitioning the spatial distribution density of a 2D point cloud set, a non-uniform mesh structure that fits the scene's geometric distribution is constructed, achieving optimized allocation and load balancing of computational resources. Furthermore, through visibility-aware training data partitioning, and based on the visibility relationships of 3D feature point trajectories, the training view is precisely matched to each sub-region, effectively solving the mismatch between the effective field of view and supervision information, ensuring that each sub-scene receives sufficient and accurate geometric supervision signals.

[0014] Step 106: Construct a 3D Gaussian splash model for each spatial sub-block. Use the allocated training image set to train the 3D Gaussian splash model in parallel. During the training process, optimize the Gaussian meta-parameters in each spatial sub-block by combining a joint loss function consisting of photometric loss, depth loss, pseudo-view geometric consistency loss, and scale regularization loss.

[0015] The 3D Gaussian splash model is an explicit reconstruction technique that uses discrete 3D Gaussian primitives as a means of scene representation. Each primitive contains physical properties that determine its spatial shape and appearance, including: center position, covariance (determining the shape and orientation of anisotropy), opacity, and color information (expressed by spherical harmonic function coefficients). During training, by introducing inverse depth regularization, pseudoview geometric consistency constraints, and scale regularization, the generation of near-field floating artifacts is effectively suppressed, guiding the model to learn accurate scene geometry, avoiding the occurrence of Gaussian points with anomalous scale, and improving the visual fidelity of the model under new perspectives.

[0016] Step 108: Merge the 3D Gaussian splash models of all trained spatial sub-blocks to generate the 3D reconstruction result of the complete scene.

[0017] Because the block boundary expansion and auxiliary point enhancement strategy is introduced in step 104, adjacent sub-blocks share the geometric information of the overlapping buffer, thus enabling seamless merging during merging and avoiding geometric distortion or reconstruction holes at the boundary.

[0018] In the aforementioned large-scale aviation city scene reconstruction method based on the divide-and-conquer strategy, firstly, addressing the uneven computational load and memory bottleneck issues caused by traditional uniform partitioning, this application proposes a scene partitioning strategy based on spatial point cloud density awareness in steps S2-S3: by normalizing the initial point cloud coordinate system and projecting it onto the horizontal plane, an adaptive KD-Tree algorithm based on point cloud density constraints is used for recursive partitioning, automatically generating finer-grained sub-blocks in dense (geometrically complex) point cloud regions and coarser-grained sub-blocks in sparse regions. This technique directly solves the load imbalance problem caused by ignoring the non-uniform distribution of point clouds in existing technologies, ensuring a balance in the number of Gaussian elements within each sub-block. This maximizes resource utilization for multi-GPU parallel training under limited memory conditions, breaking through the single-card memory limit and providing sufficient sampling density for the model to represent complex details, fundamentally alleviating the contradiction between memory bottlenecks and limited representation capabilities. Secondly, to address the geometric breaks and fusion artifacts caused by the mismatch between the view area and the supervision image in the boundary region after block segmentation, this application proposes a block training data partitioning strategy based on visibility perception in step S4: by calculating two quantitative indicators, view coverage and block coverage, training images are precisely selected for each sub-block, eliminating invalid views that only observe edge feature points; simultaneously, boundary dilation and auxiliary point enhancement strategies are introduced, adding the background point clouds of adjacent sub-blocks as auxiliary anchor points to the initialization set of the current sub-block. This technique ensures that each sub-scene can obtain a geometric supervision signal that accurately matches its spatial range, solving the mismatch problem caused by view frustum truncation where the model is forced to fit cross-regional background pixels with edge primitives, thereby guaranteeing the geometric continuity and reconstruction integrity at the sub-block boundaries. Finally, to address the issues of near-field floating artifacts and geometric distortion in the model, this application designs a scale regularization term and a pseudo-view geometric consistency constraint in step S5. On one hand, an inverse depth regularization term is introduced, utilizing the dense depth prior provided by the monocular depth estimator to supervise the rendering depth. On the other hand, virtual camera views are sampled within the scene, and a distillation loss is constructed to transfer scene structure knowledge by calculating the Pearson correlation coefficient between the rendered pseudo-view depth map and the depth estimation reference depth map. Simultaneously, a scale regularization term is added to penalize the product of Gaussian primitive volumes, forcing primitives to remain compact. This technique imposes constraints on the model from three dimensions: geometric prior, cross-view consistency, and primitive morphology. This effectively suppresses the generation of near-field floating artifacts, avoids the appearance of Gaussian points with scale anomalies, and significantly improves the geometric fidelity and visual rendering quality of the model under new perspectives. In summary, this application, through the synergistic effect of the above three innovative techniques, addresses the three major technical challenges of memory resource bottlenecks, boundary artifact breaks, and geometric distortion, ultimately achieving efficient and high-quality large-scale urban scene 3D reconstruction.

[0019] In one embodiment, coordinate system normalization is performed on the initial point cloud to obtain a two-dimensional point cloud set, including: Calculate the average viewing direction vector for all viewpoints, and construct a rotation matrix to rotate the entire scene so that the average viewing direction is aligned with the vertical axis of the preset reference world coordinate system, thus obtaining the aligned scene points; The aligned scene point cloud is projected onto a horizontal plane to obtain a two-dimensional point cloud set for spatial partitioning.

[0020] Specifically, since the original COLMAP reconstruction is based on a relative coordinate system, the generated point cloud often has a random rotational deviation from the horizontal plane of the world coordinate system. If it is directly projected onto a two-dimensional plane without alignment, it will lead to geometric distortion and spatial stretching, thereby destroying the authenticity of the point cloud density distribution.

[0021] First, a globally unified reference world coordinate system is established. Then, by analyzing the camera pose data, the average viewing direction vector of all viewpoints is calculated, and a rotation matrix is ​​constructed to rotate the entire scene so that the main direction is aligned with the y-axis (vertical axis) of the reference world coordinate system. Subsequently, the point cloud is projected onto the horizontal plane, i.e., the xz plane, and the spatial points are divided by calculating the point cloud density on the horizontal plane.

[0022] By aligning the line of sight with the vertical axis, the scene is corrected to a horizontal state, allowing it to be projected onto the horizontal plane without distortion. This lays the foundation for subsequent spatial partitioning based on point cloud density on the horizontal plane. It resolves the geometric distortion problem caused by coordinate system inconsistency, ensuring that spatial partitioning based on point cloud density accurately reflects the true spatial structure of the scene.

[0023] In one embodiment, adaptive spatial partitioning of the spatial distribution density of the two-dimensional point cloud set includes: An adaptive KD-Tree spatial partitioning algorithm based on point cloud density constraints is adopted. The main axis with the most discrete data distribution is selected by calculating the standard deviation of the projection of the point cloud onto each axis of the horizontal plane, and the median of the point cloud projection on the main axis is used as the partitioning benchmark for recursive partitioning. The recursive partitioning is controlled by the dual constraints of local point cloud density and tree depth. When the number of point clouds in the region is lower than the preset threshold or the tree depth reaches the upper limit, the partitioning is terminated, generating multiple spatial sub-blocks.

[0024] Specifically, the algorithm selects the principal axis with the most discrete data distribution by calculating the standard deviation of the axial projection, and recursively divides the data using the median of the point cloud projection on that axis as the segmentation benchmark, thereby maximizing the balance of point cloud quantity among sub-regions. Furthermore, to prevent the segmented sub-regions from exhibiting extremely elongated geometric shapes due to outliers at the edges, the algorithm introduces a boundary truncation constraint mechanism to limit the range of segmentation ratios.

[0025] like Figure 2As shown, after obtaining the minimum bounding box, Principal Component Analysis (PCA) is used to estimate the principal axis orientation of the point cloud and perform rotation correction to correct the scene's bounding box to an axis-aligned state. The specific steps are as follows: The 3D point set... Project to A two-dimensional point set is obtained from a plane. First, its covariance matrix is ​​calculated. Second, eigenvalue decomposition is performed on the covariance matrix, and the eigenvector corresponding to the largest eigenvalue is extracted as the principal axis direction of the scene. Finally, the relationship between this principal axis vector and the world coordinate system is calculated. The included angle of the axis, constructing the circumference The rotation matrix of the axes is calculated and applied to all point clouds and camera poses. After this processing, the scene's bounding box is corrected to an axis-aligned state.

[0026] The recursive process is controlled by both local point cloud density and tree depth: the partitioning terminates when the number of point clouds in the region falls below a preset threshold or the tree depth reaches its upper limit. This method ultimately adaptively constructs a non-uniform mesh structure that fits the scene's geometric distribution, achieving optimized allocation of computational resources. The specific flow of the pseudocode for the point cloud density-based adaptive scene partitioning algorithm is shown in Table 1.

[0027] Table 1

[0028] like Figure 3 As shown, the final generated scene block bounding boxes exhibit significant spatial adaptive characteristics in their spatial distribution: in densely built-up, geometrically complex city center areas, the block granularity is finer; while in areas with sparse vegetation or buildings, the block granularity adaptively increases. This achieves a reasonable allocation of computing resources, avoiding fragmented waste of computing resources, and providing standardized subtasks with approximately equal computational costs for parallel training. Through the above strategy, the originally complex large-scale scene reconstruction task has been divided into multiple standardized subtasks with approximately equal computational costs, providing a clear spatial geometric benchmark for the accurate allocation of training views based on visibility, and laying the data foundation.

[0029] In one embodiment, each spatial sub-block is partitioned based on visibility awareness training data, including: For the current spatial sub-block, the 3D-2D correspondence obtained by the motion recovery structure reconstruction is used to retrieve all candidate image sets of observed 3D points within the current spatial sub-block; For each image in the candidate image set, calculate view coverage and / or block coverage, where view coverage measures the degree of attention a single image pays to the current spatial sub-block, and block coverage measures the degree of contribution a single image makes to the geometry of the current spatial sub-block. Images that meet the preset view coverage threshold or block coverage threshold are included in the training image set of the current spatial sub-block.

[0030] Specifically, for a given spatial block First, using the 3D-2D correspondence reconstructed from COLMAP, a candidate image set of all observed 3D points within the region is retrieved. Then, to eliminate invalid views that only observe block edges or a very small number of feature points, this application defines two key screening metrics: view coverage. With block coverage View coverage is defined as the ratio of the number of feature points falling within the current sub-block to the total number of feature points observed in the image; block coverage is defined as the ratio of the number of feature points falling within the current sub-block to the total number of feature points in the current sub-block. The former measures the degree of attention a single image pays to the current block (i.e., how much content in the image belongs to the block), while the latter measures the contribution of the image to the block's geometry (i.e., how many points in the block are observed by the image). Only when a candidate image meets any of the above coverage threshold conditions will it be included in the final training set of that sub-block. The specific implementation process is shown in Table 2.

[0031] Table 2

[0032] The above process can eliminate invalid views that only observe the edges of the blocks or a very small number of feature points, ensuring that each sub-block receives sufficient and accurate geometric supervision signals.

[0033] In one embodiment, the spatial bounding box of each spatial sub-block is extended outward by a specific proportion to generate an overlapping buffer; point clouds that originally belonged to adjacent sub-blocks and were located within a specific distance threshold of the current sub-block's expansion boundary are retrieved within the field of view and added as auxiliary anchor points to the initial point cloud set of the current sub-block.

[0034] Specifically, in aerial scene segmentation, the discrepancy between the view frustum range and the spatial segmentation boundary can lead to a mismatch between the effective field of view and the supervisory information, such as... Figure 4As shown, a training image assigned to sub-region 1 extends its view frustum into the geometric space of the adjacent sub-region 2, resulting in the image containing a large amount of distant background content belonging to sub-region 2. Since the Gaussian primitives of sub-region 1 are spatially confined within the red box, they cannot cover this cross-regional background. When calculating the photometric loss of both, the model is forced to attempt to fit these pixels lacking geometric support using primitives at the edges of sub-region 1. These primitives are stretched to positions detached from the actual surface during parameter updates, or assigned incorrect color and opacity attributes, in an attempt to compensate for pixel errors that should be explained by adjacent sub-regions. This mismatch between the supervision signal and representational capability accumulates during optimization iterations, eventually leading to geometric distortion or reconstruction holes at sub-region boundaries. To address this issue, the spatial bounding box of each sub-block is first extended outward by a specific proportion. This dilation operation introduces an additional overlap buffer, allowing adjacent sub-blocks to share geometric information of their edge regions. Secondly, by retrieving the background point cloud that originally belonged to adjacent blocks within the view frustum, it is added as an auxiliary anchor point to the initialization set of the current sub-block. However, blindly introducing background point clouds across the entire field of view can have negative effects: points too far from the current sub-block contribute little to local geometric reconstruction and are prone to introducing outlier noise, interfering with the stability of the optimization gradient. Therefore, this application only selects neighboring points located within a specific distance threshold (within 1.5 times the extended boundary) of the current sub-block's expansion boundary as auxiliary anchor points. This strategy effectively shields far-field noise while supplementing necessary geometric context information, ensuring that Gaussian elements grow only within effective geometric support regions, thus achieving a balance between initialization quality and optimization stability.

[0035] While supplementing necessary geometric context information, it effectively shields far-field noise interference, ensuring that Gaussian elements grow only within the effective geometric support region. This solves the problem of mismatch between the effective field of view and supervision information, and avoids geometric distortion or reconstruction holes at the boundaries of sub-regions.

[0036] In one embodiment, the process of constructing the depth loss includes: A monocular depth estimator is used to predict the depth of the training image to obtain a predicted depth map. Using sparse 3D points generated by motion-reconstructed structures as geometric anchors, the predicted depth map is physically scaled to obtain a reference depth map. Generate a rendering depth map of the current viewpoint through volume rendering; Calculate the L1 loss between the rendered depth map and the reference depth map, and use it as the depth loss.

[0037] Specifically, in aerial photography scenarios, since the views are mostly taken from above, lacking supervision information from other angles, the model is prone to overfitting the training images, manifesting as floating points appearing near the camera in the air. To address this issue, this application introduces an inverse depth regularization term to suppress floating point generation. Building upon this, it further proposes pseudo-view geometric consistency. By enhancing the geometric constraints of new perspectives, it achieves more accurate reconstruction of the geometric structure of complex scenes.

[0038] First, depth map prediction is performed on the training images using a pre-trained monocular depth estimator, DepthAnythingV2. The predicted depth map provides dense geometric priors, but its output is typically a relative depth, lacking a true physical scale, and needs to be aligned with the scale of the point cloud reconstructed from COLMAP. Specifically, for each training view... First, sparse 3D points generated during COLMAP reconstruction are used as geometric ground truth anchor points. These 3D points are projected onto the camera coordinate system to obtain the depth values ​​corresponding to the 3D points. Then, these depth values ​​are inverted to obtain a sparse inverse depth set. Simultaneously, a monocular depth estimator is used to predict... Dense inverse depth map The predicted value set corresponding to the pixel position of the sparse inverse depth point is obtained by bilinear interpolation sampling. There is usually a linear mapping relationship between these two, that is... ,in These are the scaling transformation coefficients. This represents the offset coefficient. Considering the potential outlier noise in sparse point clouds, this application employs a robust median statistic for... and Perform parameter estimation:

[0039] in This represents the absolute deviation of the median. Using the calculated... and right By performing an affine transformation, the relative depth can be calibrated into a dense inverse depth map with absolute physical scale. :

[0040] It is worth noting that this application does not use the original depth, but the reciprocal of the depth. This is not only because the inverse depth has a linear relationship with parallax, but also to improve the numerical stability of the distant area.

[0041] Subsequently, a rendering inverse depth map of the current viewpoint is generated through volume rendering. Through calculation and of Loss calculation using depth regularization: .

[0042] Dense geometric prior supervision was introduced to effectively guide the model to learn accurate scene geometry and suppress the generation of near-field floating artifacts.

[0043] In one embodiment, the process of constructing the pseudo-view geometric consistency loss includes: Sample the virtual camera view within the scene, and use the current model to render and generate a pseudo-view RGB image and a pseudo-view depth map from the virtual camera view. A monocular depth estimator is used to predict the depth of the pseudo-view RGB image and perform physical scale recovery to obtain a reference pseudo-view depth map. Calculate the Pearson correlation coefficient between the pseudo-view depth map and the reference pseudo-view depth map, and use the L1 norm of the Pearson correlation coefficient as the pseudo-view geometric consistency loss.

[0044] Specifically, relying solely on depth supervision of the training viewpoint is insufficient to completely eliminate floating objects from a new perspective. Therefore, this application introduces pseudo-view geometric consistency constraints. Specifically, virtual cameras are sampled near the ground. Render using the current 3DGS model Pseudo-view RGB image from a certain perspective Inverse depth map Use DepthAnythingV2 to render images. Make predictions and recover the physical scale through the aforementioned steps to obtain... Then calculate and The Pearson correlation coefficient (PCorr) is used as the loss function for the geometric consistency of the pseudo-view:

[0045] The formula for calculating the Pearson correlation coefficient is as follows:

[0046] Essentially, It is a distillation loss that uses the prior scene structure information of DepthAnythingV2 as a discriminator to transfer the scene structure knowledge learned by the large-scale monocular depth estimation model to the 3DGS optimization process, constraining 3DGS to generate geometric structures that conform to the statistical laws of natural scenes.

[0047] By enhancing the geometric constraints of the new perspective, the 3DGS is constrained to generate geometric structures that conform to the statistical laws of natural scenes, effectively eliminating floating artifacts under the new perspective and achieving more accurate restoration of the geometric structure of complex scenes.

[0048] In one embodiment, the process of constructing the scale-regularized loss includes: Obtain the scaling factor of all Gaussian elements in the current sub-block along each principal axis; calculate the product of the scaling factors of each Gaussian element along each principal axis, and average the products of all Gaussian elements. Use the average value as the scaling regularization loss.

[0049] Specifically, in order to avoid reconstructing overly inflated Gaussian elements that could occlude the background or fill holes, this application introduces a scale regularization constraint, which forces the Gaussian elements to remain compact by penalizing the product of Gaussian volumes.

[0050] in, Represents the set of Gaussian elements within the current sub-block. Indicates the first The first Gaussian Yuan in the Scaling factor in each principal axis direction.

[0051] By penalizing the product of Gaussian volumes, Gaussian primitives are forced to remain compact, thus preventing the reconstruction of over-expanded Gaussian primitives that could obscure the background or fill holes.

[0052] In one embodiment, the process of constructing the photometric loss includes: A rendered image of the current viewpoint is generated through volume rendering; the corresponding real training image is obtained; and the weighted sum of L1 color loss and D-SSIM structure loss between the rendered image and the real training image is calculated as the photometric loss.

[0053] Specifically, the basic network optimization function calculates the photometric loss. get:

[0054] in Rendered image for the current viewpoint. These are real aerial training images. By calculating the difference between the two The weights are composed of a weighted average of color loss and D-SSIM loss, with the weights... Set it to 0.2. The loss forces the model to accurately fit the color values ​​of pixels, while the D-SSIM loss helps to preserve the structural texture details of the image. This loss function achieves pixel-level and structural-level consistency constraints between the rendered image and the real image.

[0055] Combining photometric loss, the complete loss function for the training phase of this application is defined as follows:

[0056] in This is a constant hyperparameter, set to 0.01. The decay rate increases exponentially with each training round, aiming to guide geometric convergence in the early stages. It exhibits exponential growth, aiming to enhance the suppression of artifacts and the correction of details in later stages.

[0057] In one embodiment, before performing coordinate system normalization on the initial point cloud, the method further includes: using a rotating caliper algorithm to calculate the minimum bounding box of the initial point cloud to determine the effective boundary of the scene.

[0058] Specifically, the input point set is first calculated using the fast convex hull algorithm. convex hull ,make convex hull The globally optimal rectangle is a set of vertices arranged in counter-clockwise order. The minimum bounding box algorithm serves as the basis for defining the scene boundaries. The specific process of the minimum bounding box algorithm is shown in Table 3.

[0059] Table 3

[0060] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this application, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Furthermore, Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0061] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0062] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for large-scale aviation city scene reconstruction based on a divide-and-conquer strategy, characterized in that, The method includes: Acquire initial point cloud and camera pose data for a large-scale aviation city scene; perform coordinate system normalization processing on the initial point cloud to obtain a two-dimensional point cloud set; The spatial distribution density of the two-dimensional point cloud set is adaptively partitioned to divide the complete scene into multiple spatial sub-blocks; training data is partitioned for each spatial sub-block based on visibility perception, and a corresponding training image set is assigned to each spatial sub-block; A 3D Gaussian splash model is constructed for each spatial sub-block. The 3D Gaussian splash model is trained in parallel using an allocated training image set. During the training process, the Gaussian meta-parameters in each spatial sub-block are optimized by combining a joint loss function consisting of photometric loss, depth loss, pseudo-view geometric consistency loss, and scale regularization loss. Merge all trained spatial sub-blocks of the 3D Gaussian splash model to generate the complete 3D reconstruction result of the scene.

2. The method according to claim 1, characterized in that, The initial point cloud is subjected to coordinate system unification processing to obtain a two-dimensional point cloud set, including: Calculate the average viewing direction vector for all viewpoints, and construct a rotation matrix to rotate the entire scene so that the average viewing direction is aligned with the vertical axis of the preset reference world coordinate system, thus obtaining the aligned scene points; The aligned scene point cloud is projected onto a horizontal plane to obtain a two-dimensional point cloud set for spatial partitioning.

3. The method according to claim 1, characterized in that, Adaptive spatial partitioning of the spatial distribution density of the two-dimensional point cloud set includes: An adaptive KD-Tree spatial partitioning algorithm based on point cloud density constraints is adopted. The main axis with the most discrete data distribution is selected by calculating the standard deviation of the projection of the point cloud onto each axis of the horizontal plane, and the median of the point cloud projection on the main axis is used as the partitioning benchmark for recursive partitioning. The recursive partitioning is controlled by the dual constraints of local point cloud density and tree depth. When the number of point clouds in the region is lower than a preset threshold or the tree depth reaches the upper limit, the partitioning is terminated, generating multiple spatial sub-blocks.

4. The method according to claim 1, characterized in that, The training data for each spatial sub-block is divided based on visibility awareness, including: For the current spatial sub-block, the 3D-2D correspondence obtained by the motion recovery structure reconstruction is used to retrieve all candidate image sets of observed 3D points within the current spatial sub-block; For each image in the candidate image set, calculate view coverage and / or block coverage, wherein the view coverage is used to measure the degree of attention a single image pays to the current spatial sub-block, and the block coverage is used to measure the degree of contribution of a single image to the geometry of the current spatial sub-block; Images that meet the preset view coverage threshold or block coverage threshold are included in the training image set of the current spatial sub-block.

5. The method according to claim 4, characterized in that, The method further includes: Extend the bounding box of each spatial sub-block outward by a specific proportion to create an overlapping buffer; retrieve the point cloud within the field of view that originally belonged to the adjacent sub-block and was located within a specific distance threshold of the current sub-block's expansion boundary, and add it as an auxiliary anchor point to the initial point cloud set of the current sub-block.

6. The method according to claim 1, characterized in that, The process of constructing the depth loss includes: A monocular depth estimator is used to predict the depth of the training image to obtain a predicted depth map. Using sparse 3D points generated by motion recovery structure reconstruction as geometric anchor points, the predicted depth map is physically scaled to obtain a reference depth map. Generate a rendering depth map of the current viewpoint through volume rendering; Calculate the L1 loss between the rendered depth map and the reference depth map, and use it as the depth loss.

7. The method according to claim 1, characterized in that, The process of constructing the pseudo-view geometric consistency loss includes: Sample the virtual camera view within the scene, and use the current model to render and generate a pseudo-view RGB image and a pseudo-view depth map from the virtual camera view. The depth of the pseudo-view RGB image is predicted using a monocular depth estimator, and physical scale recovery is performed to obtain a reference pseudo-view depth map. Calculate the Pearson correlation coefficient between the pseudo-view depth map and the reference pseudo-view depth map, and use the L1 norm of the Pearson correlation coefficient as the pseudo-view geometric consistency loss.

8. The method according to claim 1, characterized in that, The process of constructing the scale-regularized loss includes: Obtain the scaling factor of all Gaussian elements in the current sub-block along each principal axis; calculate the product of the scaling factors of each Gaussian element along each principal axis, and average the products of all Gaussian elements, using the average value as the scaling regularization loss.

9. The method according to claim 1, characterized in that, The process of constructing the photometric loss includes: A rendered image of the current viewpoint is generated through volume rendering; the corresponding real training image is obtained; and the weighted sum of L1 color loss and D-SSIM structure loss between the rendered image and the real training image is calculated as the photometric loss.

10. The method according to claim 1, characterized in that, Before performing coordinate system normalization on the initial point cloud, the method further includes: using a rotating caliper algorithm to calculate the minimum bounding box of the initial point cloud to determine the effective boundary of the scene.

Images