A hierarchical three-dimensional modeling method and system with scene skeleton as anchor point

Abstract

The application discloses a hierarchical three-dimensional modeling method and system taking a scene skeleton as an anchor point, and belongs to the technical field of computer vision and three-dimensional reconstruction. k The local subgraph is constructed and the local community is divided based on the abutment screening structure; the island-level graph is constructed based on the local community, the skeleton subgraph is constructed through the maximum weight edge disjoint spanning tree merging and the selective verification of the double-view disambiguator; the non-skeleton image is attached to the skeleton node in the corresponding community as a leaf node; and the scene skeleton is reconstructed based on the skeleton subgraph, and the complete reconstruction is completed by gradually absorbing the leaf node. Through the hierarchical graph construction and the selective verification mechanism, the application solves the problem that the robustness and the efficiency are difficult to be considered in the large-scale scene reconstruction, can effectively inhibit the visual ambiguity propagation while significantly reducing the calculation overhead, and realizes the high-integrity three-dimensional modeling.

Description

Technical Field

[0001] This invention belongs to the fields of 3D reconstruction and computer vision technology, and specifically relates to a hierarchical 3D modeling method and system with scene skeleton as anchor point. Background Technology

[0002] Motor restoration structure from Sparse Motion Mapping (SfM) is a core technology for large-scale scene sparse 3D reconstruction, widely used in embodied intelligent robots, autonomous driving, scene mapping, robot visual localization, and image neural rendering. With the development of systems such as COLMAP and GLOMAP, SfM can adapt to diverse acquisition conditions and dataset sizes, and sparse maps built based on COLMAP have become a reference model for visual localization and robot-related benchmarks.

[0003] Please see Figure 2 Currently, mainstream structure of motion (SfM) technology faces a dilemma when dealing with large-scale real-world scenarios: it is impossible to achieve both "efficiency" and "robustness." The specific technical problems are reflected in the following two core dimensions: 1. Global structure propagation failure under strong visual ambiguity (robustness problem): In scenes containing numerous repetitive textures (such as dense bookshelves or glass curtain walls), symmetrical structures, or weakly textured regions, traditional SfM methods are highly prone to mismatches. While existing deambiguation methods (such as Doppelgangers++) are robust, they typically require pairwise discrimination of all candidate image pairs in the scene graph. Since mismatches often satisfy dual-view geometric constraints and are validated by RANSAC, these erroneous connections spread globally along the scene graph like a virus. Once erroneous cross-community connections (bridging edges) are introduced, subsequent incremental reconstructions can lead to catastrophic global structural collapse (such as incorrect scene folding or mirroring). Current technologies lack a mechanism to accurately identify and intercept these most harmful cross-regional erroneous connections at the outset of global topology construction.

[0004] 2. Redundant constraints leading to an explosion in computational complexity (efficiency issue): As the number of input images increases, the scene graph constructed by traditional SfM quickly becomes highly dense. The computational complexity of backend bundle adjustment (BA) approaches the maximum complexity as the number of variables (camera pose and 3D points) increases. Existing graph sparsification methods (such as CamTrip) improve efficiency by removing redundant edges, but often employ a radical "one-size-fits-all" approach, disrupting the global connectivity of the scene. This leads to large-scale scenes being incorrectly split into multiple isolated parts, losing crucial cross-ambiguity region connections, and failing to generate complete reconstruction results. Current techniques fail to find a balance between removing invalid redundant edges to reduce computational burden and preserving key global connections to maintain scene integrity.

[0005] In summary, there is an urgent need for a hierarchical modeling method that can adaptively distinguish between local details and the global skeleton, and accurately protect and verify the key connections that determine the scene topology. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a hierarchical 3D modeling method and system with scene skeleton as anchor point to address the shortcomings of the prior art. This method and system solve the technical problems of existing SfM technology, such as easy error reconstruction of repetitive / symmetric structure scenes, excessive computational overhead of large-scale image sets, and inability to balance robustness and efficiency. The invention achieves 3D reconstruction with strong robustness, high efficiency, and high integrity.

[0007] The present invention adopts the following technical solution: A hierarchical 3D modeling method using a scene skeleton as anchor points includes the following steps: S1. A weighted scene graph based on the input, using top- k The adjacency filtering strategy constructs a local subgraph, and the connected components on the local subgraph are calculated to obtain local communities; S2. Based on local communities, bridge edges are determined and island-level graphs are constructed. On the island-level graphs, skeleton subgraphs are constructed by merging disjoint spanning trees with the maximum weight edges. The bridge edges in the skeleton subgraphs are selectively verified using a dual-view deambiguer. S3. Take the images that have not entered the skeleton subgraph as leaf nodes and attach the leaf nodes to the skeleton nodes in the corresponding local community. S4. Reconstruct the scene skeleton based on the skeleton subgraph, and gradually absorb the leaf nodes through registration, triangulation and local binding adjustment to complete the complete 3D reconstruction.

[0008] Preferably, in step S1, the top- k The adjacency filtering strategy constructs a local subgraph, specifically: for each image, only the strongest edges sorted by edge weight are retained. k The edges corresponding to each of the neighbors, where... k The value is 1; the edge weight is the number of interior points of the image pair after geometric verification.

[0009] Preferably, step S2 specifically includes: The edges connecting different local communities are defined as bridging edges. An island-level graph is constructed with local communities as nodes and bridging edges as edges. The edge weights of the island-level graph inherit the number of interior nodes of the original edges. Select on the island-level map K A spanning tree with no overlap between any two edges is formed, and the set of edges with the largest total weight is retained to form the backbone. Traverse the bridge edges in descending order of edge weight, and only add bridge edges that satisfy the constraints of different connected components, island pair diversity, and dual-view deambiguer discrimination score not lower than the threshold to the spanning tree. By adding high-weight edges within a local community, the skeleton nodes within the local community are connected into a single connected subgraph, resulting in the skeleton subgraph.

[0010] Preferably, the number of spanning trees K According to the formula K =max(1,round( n / 300)) is determined, where, n This represents the total number of input images.

[0011] Preferably, the dual-view deambiguer is Doppelgangers++, and the discrimination process is implemented using a greedy Kruskal minimum spanning tree construction method.

[0012] Preferably, step S3 specifically includes: Within the local community to which a leaf node belongs, high-weight neighbors are selected for the leaf node and connected to the skeleton node first to form leaf attachment edges; the hierarchical reconstruction graph is composed of the edges of the skeleton subgraph and the leaf attachment edges.

[0013] Preferably, step S4 specifically includes: The incremental motion recovery structure solver is restricted to run on the skeleton subgraph to recover the global scene skeleton reconstruction results; Based on 2D The 3D correspondence and PnP solver are registered with leaf images, and local binding adjustments are performed on newly added cameras and 3D points. Duplicate registration triangulation The local optimization process involves performing global binding adjustments after all images have been absorbed to obtain the final reconstruction result.

[0014] Preferably, the weighted scene graph is constructed using a standard feature extraction, matching, and geometric verification process, and the scene graph is represented as follows: G = ( V , E ),in, V Given a set of input images, E Image edges that are preserved through geometric verification.

[0015] Secondly, embodiments of the present invention provide a hierarchical 3D modeling system with a scene skeleton as an anchor point, comprising: A partitioning module is used for weighted scene graphs based on the input, employing top-... k The adjacency filtering strategy constructs a local subgraph, and the connected components on the local subgraph are calculated to obtain local communities; The skeleton module, connected to the partitioning module, is used to determine bridging edges based on local communities and construct an island-level graph. On the island-level graph, a skeleton subgraph is constructed by merging the maximum weight edge non-intersecting spanning trees. The bridging edges in the skeleton subgraph are selectively verified using a dual-view deambiguer. The node module, connected to the skeleton module, is used to take images that have not entered the skeleton subgraph as leaf nodes and attach the leaf nodes to the skeleton nodes in the corresponding local community. The reconstruction module, connected to the node module, is used to reconstruct the scene skeleton based on the skeleton subgraph. It gradually absorbs leaf nodes through registration, triangulation, and local binding adjustments to complete the full 3D reconstruction.

[0016] Preferably, the skeleton module includes: The island-level graph construction submodule is used to determine bridging edges and construct the island-level graph; The spanning tree merging submodule is used to select on an island-level graph. K The main skeleton is obtained by merging two non-overlapping spanning trees. The selective verification submodule is used to discriminate and verify bridge edges using a dual-view deambiguer; The island connectivity completion submodule is used to supplement high-weight edges within a local community, connecting skeleton nodes into a single connected subgraph.

[0017] Preferably, the reconstruction module includes: The scene skeleton reconstruction submodule is used to restore the global scene skeleton on the skeleton subgraph; The leaf absorption submodule is used to register leaf images through the PnP solver and perform local triangulation and local bundling adjustments. The global optimization submodule is used to perform global bundled adjustments on all reconstructed models.

[0018] Thirdly, a computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the hierarchical 3D modeling method described above, which uses a scene skeleton as an anchor point.

[0019] Fourthly, embodiments of the present invention provide a computer-readable storage medium including a computer program, which, when executed by a processor, implements the steps of the above-described hierarchical 3D modeling method with a scene skeleton as the anchor point.

[0020] Fifthly, a chip includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the hierarchical 3D modeling method described above, which uses a scene skeleton as an anchor point.

[0021] In a sixth aspect, embodiments of the present invention provide an electronic device, including a computer program, which, when executed by the electronic device, implements the steps of the above-described hierarchical 3D modeling method with a scene skeleton as the anchor point.

[0022] Compared with the prior art, the present invention has at least the following beneficial effects: A hierarchical 3D modeling method using a scene skeleton as an anchor point is proposed. Steps S1 and S2 simplify the complex global scene graph into a hierarchical structure composed of local communities and key bridging edges, significantly reducing the variable scale of backend optimization and improving computational efficiency. Secondly, by explicitly constructing the scene skeleton, the key connections that determine the global topology are extracted, providing a stable anchor point for subsequent incremental reconstruction and effectively suppressing the cumulative propagation of local errors. Finally, this hierarchical architecture allows the system to prioritize data with the greatest impact on the global structure, ensuring that when facing visually ambiguous scenes such as repetitive textures or symmetrical structures, the reconstruction process can prioritize locking the correct global topology, avoiding the global structural collapse caused by local mismatches in traditional methods, and achieving a balance between high integrity and high stability.

[0023] Furthermore, by setting k =1, which achieves a significant reduction in graph sparsity, retaining only the most representative local strong connections. This not only eliminates a large number of weak connections and redundant edges, greatly compressing the graph size for subsequent calculations and reducing computational complexity, but also effectively filters out potential mismatches within local communities.

[0024] Further, select K The non-intersecting spanning tree not only ensures the compactness of the skeleton graph but also introduces controlled redundancy, meaning that multiple independent paths exist between different local communities, enhancing the graph's connectivity robustness. Rigorous validation is only performed on bridging edges that cross different communities, as these are the locations where visual ambiguity is most likely to occur and is most harmful. By concentrating expensive deambiguation computations on these critical edges, optimal allocation of computational resources is achieved.

[0025] Furthermore, with the number of images n As the number of elements increases, the complexity of the scenario and the potential sources of ambiguity also increase. K A fixed value may lead to insufficient skeleton redundancy and loss of connectivity in large-scale scenarios. This formula makes KThe value grows linearly with the data size, ensuring that the skeleton graph maintains sufficient global redundancy to preserve connectivity even in large-scale scenarios. Meanwhile, setting the lower bound to 1 ensures simplicity in small-scale scenarios. This adaptive mechanism strikes an optimal balance between reconstruction integrity and computational efficiency, enabling the system to maintain excellent performance on datasets ranging from hundreds to thousands of images.

[0026] Furthermore, DG++ was chosen as the discriminator, leveraging its powerful deep learning-based discriminative capabilities to accurately identify visual ambiguities caused by repetitive textures and lighting variations. Combined with a greedy Kruskal construct, efficient algorithmic integration was achieved: discrimination is performed in real-time during the spanning tree construction process, and edges with scores below a threshold are immediately removed, preventing the addition of invalid edges. This pipeline design, which verifies while constructing, avoids wasted computation in intermediate steps compared to methods that first construct the entire graph and then remove erroneous edges. It ensures that the generated skeleton subgraph not only meets connectivity requirements in its topological structure but also has extremely high confidence in geometric vision, thus providing absolutely reliable geometric anchors for subsequent reconstruction.

[0027] Furthermore, the strategy of prioritizing connections to skeleton nodes allows leaf images to be directly anchored to rigorously validated stable structures. This provides high-quality 2D-3D correspondences for PnP pose estimation, significantly improving the registration success rate. Simultaneously, this constrained attachment strategy reduces the coupling of local bundled adjustments, making the absorption process of leaf nodes fast and stable. It achieves an efficient fusion of global skeleton-based and local community-based approaches, completing detailed completion without increasing the global computational burden.

[0028] Furthermore, firstly, the initial reconstruction is run on a compact skeleton subgraph, with far fewer variables than the entire graph, enabling rapid recovery of a globally consistent scene skeleton at extremely low cost. The subsequent registration-triangulation-local BA loop optimizes only the newly added parts, avoiding early global optimization of large-scale redundant variables and reducing time complexity from near... The efficiency is significantly reduced; the final global BA is performed only once after all images have been absorbed, at which point the model is very close to the optimal solution, resulting in extremely fast convergence. This scheduling strategy fully utilizes the prior stability provided by the skeleton, enabling the entire reconstruction process to avoid catastrophic divergence while maintaining extremely high operational efficiency.

[0029] Furthermore, the use of a standard process to construct the initial graph means that this invention does not require modification of the underlying feature extraction algorithm and can be directly integrated into existing SfM systems. The image is defined as nodes, and the number of interior points after geometric verification is defined as edge weights, which forms the basis for subsequent Top-order mapping. kThe selection and spanning tree construction provide objective and quantitative physical basis. This weight design based on the number of interior points allows the algorithm to automatically tend to retain connections with more stable geometric relationships, ensuring the input quality of hierarchical modeling from the data source and enhancing the versatility and robustness of the entire system.

[0030] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0031] In summary, this invention balances robustness and efficiency through hierarchical graph construction and selective verification mechanisms, utilizing Top-... k By significantly reducing the computational scale compared to spanning tree theory, and by performing deep deambiguation only on key bridging edges to accurately intercept global errors, fast, complete and stable 3D reconstruction is achieved in large-scale complex scenes.

[0032] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the modeling of the present invention; Figure 2 A diagram illustrating the two major challenges facing SfM; Figure 3 This is a schematic diagram of the four stages of the hierarchical 3D modeling process of the present invention; Figure 4 This is a schematic diagram illustrating the matching of the number of inliers and the top-k neighbors for the cup image dataset. Figure 5 This is a skeleton subgraph; Figure 6 This is a schematic diagram showing the reconstruction results of the method of the present invention and the comparative method on example test data; Figure 7 The ablation experiment diagram for the number of trees K; Figure 8 A schematic diagram of a computer device provided in an embodiment of the present invention; Figure 9 This is a block diagram of a chip provided according to an embodiment of the present invention.

[0034] Among them, 60. Computer equipment; 61. Processor; 62. Memory; 63. Computer program; 600. Electronic device; 610. Processing unit; 620. Storage unit; 6201. Random access memory unit; 6202. Cache memory unit; 6203. Read-only memory unit; 6204. Program / utility; 6205. Program module; 630. Bus; 640. Display unit; 650. Input / output interface; 660. Network adapter; 700. External device. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0037] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0038] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this invention generally indicates that the preceding and following objects have an "or" relationship.

[0039] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0040] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0041] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0042] This invention provides a hierarchical 3D modeling method using a scene skeleton as an anchor point. First, a skeleton graph is constructed from bottom to top, then the scene skeleton is reconstructed from top to bottom. The complete scene is gradually restored through iterative image registration and triangulation. A hierarchical 3D modeling system, HiSfM (Hierarchical Scaffold-anchored Structure-from-Motion), is proposed, a hierarchical coarse-to-fine SfM framework for visually ambiguous scenes. This method aims to simultaneously solve two core problems: first, erroneous associations caused by repetitive or symmetrical structures propagate globally along the scene graph, leading to catastrophic reconstruction errors; second, redundant images and edges significantly increase backend optimization overhead, making bundle adjustment a computational bottleneck. Therefore, this invention does not treat all images and image pairs equally into the same reconstruction process. Instead, it explicitly distinguishes between local and global structures: high-confidence, dense, and lightweight connections are retained locally to maintain stability; at the global level, only a small number of key connections that truly determine the overall topology propagation are retained, and these connections are rigorously verified. Subsequently, a coarse-to-fine reconstruction schedule is adopted. First, a stable scene skeleton is restored on a compact and reliable skeleton map. Then, local support is used to efficiently absorb the remaining image, thereby reducing the overall computational overhead while controlling error propagation.

[0043] Let the input be a scene graph constructed using a standard feature extraction, matching, and geometric verification process. Let this graph be denoted as . G = ( V , E ),in, V ={ I 1, I 2, ..., I n} represents the input image set. E This represents the image edges preserved through geometric verification. For any edge... e ij ∈ E Its weight w ij Defined as the number of interior points of the corresponding image pair after geometric verification.

[0044] Please see Figure 4 Using the first image in the dataset 'cup' as the query, this example displays the number of matching neighborhoods between the first image and other images. The top-1 and top-3 neighbors are marked in the graph to illustrate the distribution characteristics of strong local connections.

[0045] Please see Figure 1 This invention discloses a hierarchical 3D modeling method using a scene skeleton as anchor points, comprising the following steps: S1. A weighted scene graph based on the input, using top- k The adjacency filtering strategy constructs a local subgraph, and the connected components on the local subgraph are calculated to obtain local communities; The first step of HiSfM is to divide the original scene graph into several local communities, each referred to as an island. The goal of this step is not to directly recover the global structure, but rather to identify image sets with strong internal connectivity and high local consistency from the original scene graph. Intuitively, in real-world scenes, images within the same local region often have high co-sightship and stronger matching support, thus their connections are usually more reliable; conversely, connections spanning large-scale structures, especially those spanning repetitive or symmetrical regions, are more susceptible to visual ambiguity. Based on this observation, the entire graph is first decomposed into a series of locally reliable substructures, and then global connectivity is modeled and processed separately in the next stage.

[0046] Formalistically, let the input scene diagram be... G = ( V , E A simple yet effective top- k An adjacency filtering strategy is used to construct local subgraphs. Specifically, for each image... I i Only retain its right to use the border. w ij The strongest after sorting k 1 neighbor. In other words, if the image I i If a geometrically verified matching edge exists between several images, then only the first edge with the highest number of interior vertices is retained. k One edge is selected, while the remaining edges are temporarily ignored. After performing this operation on all images, a sparse subgraph is obtained, denoted as . G I = ( V , E I Compared to the original scene image, G I Instead of trying to preserve all possible connections, it consciously highlights the most representative local strong connections of each node.

[0047] Subsequently,G I We compute the connected components and treat each connected component as a local community, or an island. Let the final island division be... P ={ I 1, I 2, ..., I m}, each of which I x V To represent an island, m The number of islands. In this way, the original large-scale scene graph is naturally decomposed into several subsets with strong internal connections and stronger local geometric consistency.

[0048] There are two considerations behind adopting this strategy. First, from a matching statistics perspective, an image and its nearest neighbors often correspond to the most stable local geometric relationships, and these edges are less likely to be mismatches or ambiguous edges. Second, from a system efficiency perspective, removing a large number of weak and redundant connections in advance can significantly reduce the burden on subsequent global structure analysis and backend optimization. Especially in large-scale scenes, the original scene graph is often highly dense, with many edges providing only limited additional information but significantly increasing computational costs. By using top-k filtering, the graph size can be effectively compressed while preserving local stability.

[0049] In this invention, the following is adopted: k =1. While this setting is very simple, experiments show that it can effectively extract image communities with strong local support. This is because the strongest neighbors are usually sufficient to reflect the dominant affiliation of a node in the local space, and the resulting connected components often naturally correspond to local structural units in the scene. Compared to larger... k The top-1 algorithm can further suppress redundant edges, making the entire hierarchical structure clearer.

[0050] S2. Based on local communities, bridge edges are determined and island-level graphs are constructed. On the island-level graphs, skeleton subgraphs are constructed by merging disjoint spanning trees with the maximum weight edges. The bridge edges in the skeleton subgraphs are selectively verified using a dual-view deambiguer. After completing the local community partitioning, the next step is to construct a compact yet robust global skeleton subgraph to describe the key connectivity relationships between the islands. This step is central to HiSfM's defense against the propagation of visual ambiguity.

[0051] Please see Figure 5 By merging multiple edge-disjoint spanning trees, higher edge connectivity can be achieved on island-level graphs. For example, 2-edge connectivity means that the graph remains connected even if at most one edge is lost, thus improving the robustness of the skeleton.

[0052] The basic idea is that truly dangerous errors don't necessarily occur within local communities, but are more likely to occur on a few key edges connecting different communities. This is because once an erroneous cross-community edge enters the reconstruction process, it can potentially connect two incorrect local structures that shouldn't be connected, and this error can propagate throughout subsequent registration and optimization, ultimately leading to the collapse of the overall structure. Therefore, rather than performing expensive deambiguity verification on all candidate edges, it's more necessary to concentrate the computational budget on these critical bridging edges that "determine the global topology."

[0053] S201, Bridge Edge and Island Level Diagram Given island division P ={ I 1, I 2, ..., I m}, define all edges connecting different islands as bridging edges. That is, if an edge... e ij The two endpoints belong to different islands. I a and I b ,and a ≠ b If an edge is not a bridge edge, then that edge is considered a bridging edge. The set of all bridging edges is denoted as . B .

[0054] Based on this, we further construct an island-level business map, denoted as... G bar = ( P , B In this graph, nodes are no longer single images, but rather local islands; edges are formed by bridging edges from the original scene graph, with edge weights inherited from the number of interior vertices of the corresponding original edge. In this way, the originally complex image-level scene graph is elevated to a more abstract community-level graph representation: local consistency modeling was completed in the previous stage, and the current stage focuses on how these local communities should be reliably connected.

[0055] S202, Merging disjoint spanning trees with maximum weight edges To balance compactness and robustness at the global level, we are not satisfied with constructing only a single spanning tree connecting all islands. Instead, we aim to retain a certain degree of redundancy within a limited scope, ensuring the overall structure remains recoverable even if some key edges fail during subsequent registration. Therefore, the skeleton construction problem is formulated as the problem of merging the maximum weight disjoint spanning trees on an island-level graph.

[0056] Specifically, the hope is to achieve this on island-level maps. G bar Select aboveK A spanning tree with no overlap between its two sides T 1, T 2, ..., T K This maximizes the total weight of all selected edges. In other words, provided that "each tree covers all islands and different trees do not share edges," cross-island connections with higher interior support are prioritized. The union of all selected bridging edges forms the global backbone of the skeleton graph.

[0057] This design has two important functions. First, it ensures the skeleton graph is sufficiently compact, avoiding the reintroduction of large-scale redundant connections from the original scene graph. Second, it's not a simple single-tree structure, but rather introduces "controlled redundancy" at the global level through K edge-disjoint spanning trees. This means that multiple independent propagation paths exist between different local communities. If a key edge fails to effectively support subsequent reconstruction due to local matching instability, pose estimation failure, or other reasons, the remaining paths may still maintain overall connectivity. From a graph theory perspective, if a graph contains... K If a graph has a spanning tree with non-intersecting edges, then the graph has at least one spanning tree. K - Edge connectivity property. Intuitively, this means that deleting a small number of edges will not easily destroy the overall connectivity of the graph. For example, when K When the ratio is 2, the graph remains connected even after losing any one edge. This property is particularly important for incremental reconstruction because it provides greater fault tolerance for global propagation.

[0058] S203, Selective Disambiguation Verification Although bridging edges are highly important, they are also the most likely to contain ambiguous relationships. Therefore, a strong dual-view deambiguer—Doppelgangers++ (DG++)—is introduced during the skeleton edge selection process to verify candidate bridging edges. The key here is not in using DG++ itself, but in how it is used: DG++ is not called on all original edges, but only on those bridging edges that have the potential to enter the skeleton and thus affect the propagation of the global structure.

[0059] In implementation, a greedy, Kruskal-like minimum spanning tree construction process is used to approximate the solution of the above-mentioned edge-disjoint spanning tree merging problem. For the ... t The round-tree construction begins by initializing a disjoint-set data structure on the island-level graph to maintain currently connected island-level components. Then, bridge edges are traversed in descending order of edge weight. For each candidate edge, the following three conditions are checked: First, check if the two endpoints connected by the edge belong to different connected components in the current disjoint-set data structure. If they belong to the same component, adding the edge would create a cycle, and therefore it cannot be part of the current tree.

[0060] Second, does the island pair corresponding to the edge satisfy the diversity constraint? To do this, maintain a set... U This constraint is used to record the connection relationships of selected island pairs. Its purpose is to avoid repeatedly selecting different image-level edges between the same pair of islands across multiple trees, thereby promoting greater diversity in the coverage paths of the spanning tree, rather than repeatedly relying on the same island pairs.

[0061] Third, the edge is evaluated by DG++, meaning its ambiguity score is not lower than a given threshold τ. Only bridging edges that are considered sufficiently reliable by the depth discriminator are eligible to enter the skeleton.

[0062] Only when all three conditions above are met will the edge be added to the current spanning tree, and the corresponding two island-level connected components be merged in the disjoint-set data structure. If the current tree already contains... m When there is one edge connecting all islands, it means all islands are connected, and we can end this round of tree construction. Then we proceed to the next round to continue constructing the next non-intersecting spanning tree, until completion. K Until the end of the cycle.

[0063] By employing this strategy, HiSfM concentrates its expensive deep deambiguation budget on a small number of bridging edges that truly have global impact. On the one hand, this is far more efficient than validating all candidate image pairs; on the other hand, since these edges control the topological connections between different local communities, rigorous screening at these locations also minimizes the propagation of ambiguity.

[0064] S204, Skeleton Reinforcement and Island Connectivity Completion The set of bridging edges, consisting of the union of K non-intersecting spanning trees, already forms a backbone with global redundancy. However, in practical implementations, the backbone can be enhanced to a limited extent as needed. For example, a small number of high-weight bridging edges can be added to increase the connectivity density in certain key regions; or verified backup bridging edges can be added near low-weight connections to improve stability in weakly connected scenarios. Since these additional edges also belong to cross-island connections and may be affected by visual ambiguity, they must also be verified by DG++ before being added to the backbone.

[0065] Furthermore, it's important to note that within an island, multiple image nodes may be connected to external islands via different bridging edges. In other words, at the image level, the skeleton doesn't necessarily form a single connected structure naturally within each island. To address this, a small number of connecting edges are added within each island, connecting all island nodes participating in the skeleton into a single connected subgraph. Since these island edges originate from the strong local regions obtained from the previous top-k construction, they typically have high internal vertex support, thus eliminating the need for additional disambiguation verification.

[0066] After the above steps, a final image-level skeleton subgraph is obtained. G s = ( V s , E s ).in, E s Includes verified cross-island bridging edges and necessary in-island completion edges; V s This is the set of all image nodes involved in the skeleton edge connections. This skeleton graph is both compact and has a certain degree of redundancy, serving as the basis for subsequent global scene skeleton reconstruction.

[0067] S3. Take the images that have not entered the skeleton subgraph as leaf nodes and attach the leaf nodes to the skeleton nodes in the corresponding local community. Please see Figure 3 In the middle c, after obtaining the skeleton subgraph G s = ( V s , E s After this, the remaining nodes in the original image set that were not included in the skeleton no longer participate in the global skeleton construction, but are uniformly regarded as "leaf nodes". Let the set of leaf nodes be denoted as . V l = V \ V s The next step for HiSfM is to attach these leaf nodes to the existing skeleton in a low-cost, localized manner, thereby gradually expanding the reconstruction coverage without reintroducing a large amount of global redundancy.

[0068] The design principle at this stage is that the role of leaf nodes is primarily to complete local details and coverage, rather than to determine the global topology. Therefore, they should rely as much as possible on high-confidence connections within their own community for attachment, and not participate in new cross-community propagation. Specifically, for each island... I l Only select attached edges for leaf nodes within the island. For any leaf node... i ∈I l ∩ V l A small number of high-weight neighbors are selected from the same island, and nodes that already belong to the skeleton are connected first. The purpose of this is to: firstly, anchor the leaf nodes directly to the scene skeleton that has been restored to stability, making it easier for them to obtain sufficient and reliable geometric support for subsequent registration; if there are not enough skeleton neighbors in the island, a small number of leaf-leaf connections can be added as needed to enhance local support.

[0069] Ultimately, all the edges used for leaf attachment form a set. E l Therefore, the hierarchical reconstruction diagram of HiSfM can be denoted as: H = ( V , E H ),in E H = E s ∪ E l .here, E s Provides a global structural backbone across islands. E l This is responsible for lightly absorbing the remaining image within the island. Because... E l All edges in the array lie within locally strongly supported regions, typically eliminating the need for additional validation using DG++. This design ensures that the computational budget for disambiguation is always focused on the most critical global join locations, while local expansion remains efficient.

[0070] Algorithm 1 Skeleton Subgraph Construction enter: 1) Original weighted scene diagram G = ( V , E , w ); 2) Island division results P ={ I 1, I 2, ..., I m}; 3) Cross-island bridge junction assembly B ; 4) Number of spanning trees K ; 5) DG++ discrimination threshold τ.

[0071] Output: Skeleton subgraph G s = ( V s, E s ).

[0072] step: 1. Initialize the skeleton edge set E s It is an empty set.

[0073] 2. Initialize the collection U Empty, used to record the relationships between island pairs that have already been selected, to ensure the diversity of cross-tree edge selection.

[0074] 3. Regarding t =1 to K The following procedures are executed sequentially to construct the first... t Island-level spanning tree: 3.1 Initialize the disjoint-set data structure at the island level so that each island becomes a connected component.

[0075] 3.2 Initialize the current spanning tree edge set T t Empty.

[0076] 3.3 Traverse all bridge edges in descending order of edge weight. e ij ∈ B For each candidate edge, perform the following judgment: Let o denote that the two endpoints connected by this edge belong to the island. I a and I b The corresponding island pair is denoted as κ=( I a , I b ).

[0077] o If κ has already appeared in the set U If the edge is in the middle, then skip that edge; oIf I a and I b If the edge already belongs to the same connected component in the current union set, then skip that edge; o Calculate DG++ in image pairs ( i , j The edge is skipped if the score is less than the threshold τ. If all the above conditions are met, then the edge will be... e ij Add to current spanning tree T t And merge in the lookup set I a and Ib The two corresponding connected components.

[0078] 3.4 If the current spanning tree T t The number of sides in has reached m If 1, then stop the current iteration.

[0079] 3.5 Add the edges from the current spanning tree to the skeleton edge set, i.e., update. E s ← E s ∪ T t .

[0080] 3.6 will T t Add all edge-island pairs to the set. U This is used for diversity constraints in subsequent rounds of tree construction.

[0081] 4. After completion K After constructing the ring tree, collect all image nodes connected by the skeleton edges to obtain the node set. V s .

[0082] 5. For each island, if there are multiple skeleton nodes inside that are connected to external bridging edges, then add a small number of high-weight edges inside the island to connect these skeleton nodes into a single connected subgraph.

[0083] 6. Add these island-filling edges. E s .

[0084] 7. Output skeleton subgraph G s = ( V s , E s ).

[0085] S4. Reconstruct the scene skeleton based on the skeleton subgraph, and gradually absorb the leaf nodes through registration, triangulation and local binding adjustment to complete the complete 3D reconstruction.

[0086] Please see Figure 3 (d) After obtaining the hierarchical graph H = ( V , E HFollowing this, a two-stage incremental SfM is executed in the order of "skeleton first, leaves later". This reconstruction strategy corresponds one-to-one with the graph structure design: first, a globally stable structure is restored on the compact skeleton that has been verified by disambiguation, and then the reconstruction range is gradually expanded by relying on local strong connections. This helps to suppress the propagation of global errors and also significantly reduces the problem scale faced by early back-end optimization. Let the current sparse reconstruction result be denoted as . R = ( P , X ),in, P This represents the set of recovered camera poses. X This represents the set of three-dimensional points that have been triangulated.

[0087] S401, Scene Skeleton Reconstruction In the first stage, the existing incremental SfM solver is restricted to the skeleton subgraph. G s On the run, a global scene skeleton reconstruction is restored using only skeleton nodes and skeleton edges. R s = ( P s , X s The "scene skeleton" here is not the final complete model, but rather the most core and stable structural support in the entire scene. Because the key cross-island edges in the skeleton have been selectively deambigued, the scene skeleton has higher reliability when facing strong visual ambiguity and is less prone to large-scale mis-splitting caused by incorrect bridging edges.

[0088] On the other hand, compared with performing incremental reconstruction directly on the entire graph, building the scene skeleton on the skeleton first can significantly reduce the computational burden in the early stages. Since the skeleton graph only contains a small number of key nodes and edges, the number of variables and the scale of constraints handled by early registration, triangulation, and bundle adjustments are significantly smaller, thus enabling the establishment of a globally consistent initial structure as quickly as possible at a lower cost.

[0089] S402, Stepwise absorption of leaf nodes After obtaining a stable scene skeleton, leaf nodes are gradually incorporated into the current model. For a leaf image that has not yet been registered... i Only consider its role in hierarchical graphs H The text lists registered neighbors, primarily from the attached edges within their respective islands. It then compares these registered neighbors with the image. i The matching relationships between points can be improved by utilizing existing 3D points in the current reconstruction, transforming the original 2D–2D matching relationships into 2D–3D correspondences. Subsequently, a robust PnP solver is used to estimate the image. i The camera pose.

[0090] If the image is successfully registered, triangulation is further performed between it and adjacent registered images to recover new 3D points, and local bundle adjustments are made to optimize the new camera and related 3D points. This local BA only operates on the sub-region involved in the current expansion, thus the cost is far lower than performing large-scale global optimization at every step.

[0091] This "registration-triangulation-local optimization" process is repeated until no more leaf images can be successfully registered. Finally, after all images have been absorbed or the incremental process has converged, a global binding adjustment is performed on the entire model to obtain the final complete reconstruction result. R = ( P , X ).

[0092] This coarse-to-fine reconstruction approach differs fundamentally from traditional incremental SfM, which directly runs on the original scene graph. Traditional methods involve all candidate connections in global structure propagation from the outset, making them susceptible to interference from erroneous long-range edges. Furthermore, early binding adjustments face numerous redundant variables and constraints, resulting in high computational costs. In HiSfM, however, only a small number of validated skeleton edges truly influence the global structure, thus significantly suppressing global error propagation. Simultaneously, most redundant images are not optimized in the initial stages but are gradually absorbed by local high-confidence connections after the scene skeleton is built. This makes the reconstruction process more stable and efficient.

[0093] In summary, HiSfM's coarse-to-fine reconstruction is not simply a matter of "starting with a small amount and then expanding to a large amount," but rather it is tightly coupled with the hierarchical structure of the graph: at the global level, a compact and reliable skeleton provides structural anchoring, while at the local level, high-confidence edges within islands complete the details. This structured reconstruction strategy is the key to HiSfM's simultaneous achievement of deambiguity and efficiency.

[0094] In another embodiment of the present invention, a hierarchical 3D modeling system with a scene skeleton as the anchor point is provided. This system can be used to implement the above-mentioned hierarchical 3D modeling method with a scene skeleton as the anchor point. Specifically, the hierarchical 3D modeling system with a scene skeleton as the anchor point includes a partitioning module, a skeleton module, a node module, and a reconstruction module.

[0095] The partitioning module is used to create a weighted scene graph based on the input, employing a top-down approach. k The adjacency filtering strategy constructs a local subgraph, and the connected components on the local subgraph are calculated to obtain local communities; The skeleton module, connected to the partitioning module, is used to determine bridging edges based on local communities and construct an island-level graph. On the island-level graph, a skeleton subgraph is constructed by merging the maximum weight edge non-intersecting spanning trees. The bridging edges in the skeleton subgraph are selectively verified using a dual-view deambiguer. The skeleton module includes: The island-level graph construction submodule is used to determine bridging edges and construct the island-level graph; The spanning tree merging submodule is used to select on an island-level graph. K The main skeleton is obtained by merging two non-overlapping spanning trees. The selective verification submodule is used to discriminate and verify bridge edges using a dual-view deambiguer; The island connectivity completion submodule is used to supplement high-weight edges within a local community, connecting skeleton nodes into a single connected subgraph.

[0096] The node module, connected to the skeleton module, is used to take images that have not entered the skeleton subgraph as leaf nodes and attach the leaf nodes to the skeleton nodes in the corresponding local community. The reconstruction module, connected to the node module, is used to reconstruct the scene skeleton based on the skeleton subgraph. It gradually absorbs leaf nodes through registration, triangulation, and local binding adjustments to complete the full 3D reconstruction.

[0097] The reconstruction module includes: The scene skeleton reconstruction submodule is used to restore the global scene skeleton on the skeleton subgraph; The leaf absorption submodule is used to register leaf images through the PnP solver and perform local triangulation and local bundling adjustments. The global optimization submodule is used to perform global bundled adjustments on all reconstructed models.

[0098] This invention provides a terminal device comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, graphics processing units (GPUs), tensor processing units (TPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to achieve a corresponding method flow or function. The processor described in this embodiment can be used for the operation of a hierarchical 3D modeling method anchored by a scene skeleton, including: Based on the input weighted scene graph, a top- k An adjacency filtering strategy is used to construct a local subgraph. Connected components are calculated on the local subgraph to obtain local communities. Bridge edges are determined based on the local communities, and an island-level graph is constructed. A skeleton subgraph is built on the island-level graph using a maximum weight edge non-intersecting spanning tree merging method. A dual-view deambiguer is used to selectively verify the bridge edges in the skeleton subgraph. Images that do not enter the skeleton subgraph are used as leaf nodes, and the leaf nodes are attached to the skeleton nodes in the corresponding local communities. The scene skeleton is reconstructed based on the skeleton subgraph. Leaf nodes are gradually absorbed through registration, triangulation, and local binding adjustments to complete the full 3D reconstruction.

[0099] Please see Figure 8 The terminal device is a computer device. In this embodiment, the computer device 60 includes a processor 61, a memory 62, and a computer program 63 stored in the memory 62 and executable on the processor 61. When executed by the processor 61, the computer program 63 implements the hierarchical 3D modeling method with a scene skeleton as the anchor point as described in this embodiment. To avoid repetition, these details are not elaborated here. Alternatively, when executed by the processor 61, the computer program 63 implements the functions of each model / unit in the hierarchical 3D modeling system with a scene skeleton as the anchor point as described in this embodiment. To avoid repetition, these details are not elaborated here.

[0100] Computer device 60 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. Computer device 60 may include, but is not limited to, a processor 61 and a memory 62. Those skilled in the art will understand that... Figure 8 This is merely an example of computer device 60 and does not constitute a limitation on computer device 60. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0101] The processor 61 may be a Central Processing Unit (CPU), or other general-purpose processors, graphics processing units (GPUs), tensor processing units (TPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0102] The memory 62 can be an internal storage unit of the computer device 60, such as a hard disk or memory of the computer device 60. The memory 62 can also be an external storage device of the computer device 60, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the computer device 60.

[0103] Furthermore, the memory 62 may include both internal storage units of the computer device 60 and external storage devices. The memory 62 is used to store computer programs and other programs and data required by the computer device. The memory 62 can also be used to temporarily store data that has been output or will be output.

[0104] Please see Figure 9 The terminal device is an electronic device 600, which is manifested in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0105] The storage unit stores program code, which can be executed by the processing unit 610 to perform the steps described in the method section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 610 can perform actions such as... Figure 1 The steps are shown in the figure.

[0106] Storage unit 620 may include readable media in the form of volatile storage units, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include read-only memory (ROM) 6203.

[0107] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0108] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the multiple bus structures.

[0109] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem). This communication can be performed via input / output interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network, wide area network, and / or public network, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0110] Example 4 This invention also provides a storage medium, specifically a computer-readable storage medium, which is a memory device in a terminal device for storing programs and data. It is understood that the computer-readable storage medium here can include both built-in storage media in the terminal device and extended storage media supported by the terminal device; it can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by a processor, which can be one or more computer programs (including program code). More specific examples of the computer-readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical fiber, portable compact disk read-only memory, optical storage device, magnetic storage device, or any suitable combination thereof.

[0111] Computer-readable storage media also include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium can also be any readable medium other than a readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, radio frequency, etc., or any suitable combination thereof.

[0112] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0113] One or more instructions stored in a computer-readable storage medium can be loaded and executed by a processor to implement the corresponding steps of the hierarchical 3D modeling method with scene skeleton as anchor point in the above embodiments; one or more instructions in the computer-readable storage medium are loaded and executed by the processor in the following steps: Based on the input weighted scene graph, a top- k An adjacency filtering strategy is used to construct a local subgraph. Connected components are calculated on the local subgraph to obtain local communities. Bridge edges are determined based on the local communities, and an island-level graph is constructed. A skeleton subgraph is built on the island-level graph using a maximum weight edge non-intersecting spanning tree merging method. A dual-view deambiguer is used to selectively verify the bridge edges in the skeleton subgraph. Images that do not enter the skeleton subgraph are used as leaf nodes, and the leaf nodes are attached to the skeleton nodes in the corresponding local communities. The scene skeleton is reconstructed based on the skeleton subgraph. Leaf nodes are gradually absorbed through registration, triangulation, and local binding adjustments to complete the full 3D reconstruction.

[0114] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0115] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0116] The proposed hierarchical structure-from-motion (HiSfM) system is compared with two representative methods: CameraTriplets (CamTrip), a scene graph method that combines some disambiguation capabilities with an emphasis on efficiency; and Doppelgangers++ (DG++), a visual disambiguation method that prioritizes robustness. All methods use the same COLMAP as the SfM backend and the same feature extraction and matching results to ensure a fair comparison; the differences lie only in the selection / validation of the scene graph and the reconstruction scheduling method.

[0117] The experimental platform was a workstation equipped with an Intel i7-14700K CPU, an NVIDIA RTX 3090 GPU (24GB VRAM), and 128GB of RAM. All methods used CUDA-accelerated BA with the same BA solution settings. CamTrip parameters were used. m =0.7, DG++ uses a threshold τ=0.8, both following the original authors' recommended settings. For HiSfM, it is recommended that the number of spanning trees K in the skeleton adaptively increase with the image size, with approximately one tree configured for every 300 images, to maintain sufficient global connectivity as the data size increases.

[0118] Specifically, in the experiment, the following were taken K =max(1,round( n / 300), where, n This represents the total number of images. The impact of this setting will be further analyzed in subsequent ablation experiments. All runtimes include deambiguity inference and SfM reconstruction processes.

[0119] Evaluation on ambiguous datasets First, the performance of the method was evaluated on the classic ambiguity test dataset proposed by Heinly et al. and Yan et al., with the original COLMAP as the baseline. The quantitative results are shown in Table 1.

[0120]

[0121] Table 1 compares HiSfM with two representative methods: CameraTriplets (CamTrip), a scene graph method that combines some disambiguation capabilities with a greater emphasis on efficiency; and Doppelgangers++ (DG++), a visual disambiguation method that prioritizes robustness. All methods use the same COLMAP backend and the same feature extraction and matching results to ensure a fair comparison. The differences lie only in the selection and validation methods for the scene graphs and the reconstruction scheduling strategy.

[0122] Overall, these results clearly reflect the trade-off between robustness and efficiency. CamTrip typically runs faster on the Heinly dataset, but often registers only a small number of images, and may even incorrectly split a complete scene into multiple parts, consistent with the tendency of aggressive sparsification to lose key global connections. DG++ generally outperforms CamTrip in robustness, but its runtime increases significantly due to the need to discriminate a large number of candidate image pairs. In contrast, HiSfM achieves better reconstruction completeness while maintaining lower time overhead. This is because it performs disambiguation verification only on a small number of inter-island bridging edges and first completes global structure recovery on a compact scene skeleton before gradually attaching the remaining images, thus avoiding the high cost of exhaustive verification. Taking the Cereal dataset as an example, DG++ failed to achieve correct reconstruction, while CamTrip only registered about 56% of the images; in contrast, HiSfM successfully recovered complete and correct reconstruction results and was the fastest among the compared methods. This demonstrates that the "selective verification + representative skeleton" design can better balance robustness and efficiency.

[0123] To further evaluate geometric quality, beyond just the number of registered images, the camera poses obtained by each method were reapplied to the original, unfiltered COLMAP database. 3D points were retriangulated using the original image pairs and keypoint matching, and the total number of 3D points, average reprojection error, and average trajectory length were calculated. Since COLMAP automatically removes anomalous triangulation points based on reprojection consistency, these metrics can serve as effective proxies for pose quality under a shared triangulation process. Experimental results show that in ambiguous scenes, HiSfM's accuracy is generally comparable to, or even better than, CamTrip and DG++, while its runtime is significantly lower. For example, on the BigBen dataset, HiSfM's method is roughly equivalent to DG++ in terms of the number of registered images, but is more than 25 times faster; in contrast, although CamTrip is faster, the reconstruction results are incomplete due to the removal of too many scene edges.

[0124] Please see Figure 6 The diagram illustrates the reconstruction results of the method of this invention and the comparative method on example test data. #reg represents the number of reconstructed images, and time represents the reconstruction time.

[0125] Evaluation on common internet datasets Table 2

[0126] Table 2 shows the total number of images in the dataset ("#img"), the number of images successfully registered and participating in reconstruction ("#reg"), and the total reconstruction time (in minutes), including inference. Bold text indicates the best result, and underlined text indicates the second-best result. Red numbers in the "time" column indicate methods that take an order of magnitude longer than other methods. The upper part of the table is from the 1DSfM dataset, and the lower part, NotreDame, is from PhotoTourism.

[0127] To verify the generalization ability of the method of the present invention in more general scenarios, further experiments were conducted on larger and more complex Internet photo collections in 1DSfM and PhotoTourism, and the results are shown in Table 2.

[0128] On these general datasets, HiSfM achieves higher reconstruction integrity than CamTrip in most scenarios, while being significantly faster than DG++. DG++'s runtime is particularly impressive on large-scale image sets because it relies on extensive pairwise disambiguation processes; in scenarios with many candidate edges, this cost increases rapidly, even approaching O(n). n 2 At the O(n) level. In contrast, HiSfM performs validation only on a controlled number of globally critical bridge edges, and its size empirically follows roughly O(n) level. Kn CamTrip is generally fast, but it is prone to reduced reconstruction integrity due to excessive sparsity; in ultra-large-scale scenarios such as Trafalgar, it may even fail outright because traversing a massive number of triples is already too much to handle.

[0129] These results demonstrate that HiSfM, through its design of "compact and validated scene skeleton + efficient coarse-to-fine completion," achieves a better balance between robustness and efficiency without sacrificing integrity and geometric quality. More importantly, HiSfM is not only applicable to highly ambiguous scenes but can also be effectively generalized to ordinary internet photo collections, maintaining strong reconstruction capabilities and structural consistency even in large-scale scenes.

[0130] ablation experiment parameter K Controlling the number of non-intersecting spanning trees during skeleton construction is a crucial factor affecting global connectivity in HiSfM. Therefore, further analysis is needed. K Impact on the integrity of the reconstruction.

[0131] In the experiment, several representative scenes were selected from the ambiguous dataset by Heinly et al. and the general dataset of 1DSfM. The image sizes roughly covered four orders of magnitude: 300, 600, 900, and 1200. The parameters were then set accordingly. K HiSfM was run on the subsets {1, 2, 3, 4, 5}. The number of registered images was used as the integrity metric. The results are shown in Figure 7. Overall, with... K As the size increases, the number of registered images often increases, but the runtime also increases accordingly. K Reaching or exceeding the recommended value, i.e., max(1, round( n When the number of registered images reaches 300, the number of registered images usually stabilizes. For example, in the RomanForum scenario, [the number of registered images is likely 300]. K Increasing from 4 to 5 increased runtime by about 15%, but ultimately only registered 4 more images.

[0132] This indicates that the method used is... K Adaptive settings are already able to recover most images quite well, while further increasing skeleton redundancy brings very limited benefits but introduces additional computational costs.

[0133] Please see Figure 7 Number of trees K Ablation experiments: Each bar chart represents the number of registered images corresponding to different K values ​​in the same scene. The horizontal axis is labeled Scene (#images), i.e., the scene name and the total number of images. The number of registered images is marked near the top of the bar, and the corresponding runtime (minutes) is above it. Green shaded bars represent the recommended K value.

[0134] HiSfM first divides the scene graph into islands with strong local consistency, then constructs a compact and robust global skeleton through edge-disjoint spanning trees, and selectively verifies only the key inter-island bridging edges. Subsequently, the system first reconstructs a stable scene skeleton on the verified skeleton, and then gradually absorbs the remaining images, thereby suppressing ambiguity propagation while reducing backend optimization overhead.

[0135] Experiments on ambiguous benchmarks and large-scale internet photo datasets show that HiSfM has significantly lower time costs than exhaustive deambiguation methods, while also outperforming radical graph sparsification methods in terms of reconstruction integrity, enabling more robust and complete 3D reconstruction.

[0136] In summary, this invention presents a hierarchical 3D modeling method and system anchored by a scene skeleton. Through hierarchical scene graph construction, selective disambiguation verification, and coarse-to-fine progressive reconstruction, it fundamentally solves the problems of traditional SfM technology, such as easy reconstruction failure in visually ambiguous scenes, low computational efficiency on large-scale image sets, and difficulty in balancing robustness and integrity. This invention performs disambiguation verification only on key bridging edges, accurately blocking the global propagation of incorrect matches and significantly improving reconstruction robustness in repetitive and symmetrical structural scenes. Through hierarchical sparsity and controlled redundancy design, it greatly reduces redundancy constraints and bundling adjustment overhead, achieving up to 17 times faster operation than similar full-scale disambiguation methods, and efficiently adapting to ultra-large-scale image datasets. Compared with radical sparsification methods, this invention maintains a higher image registration rate and scene integrity, without scene splitting or structural loss. It is compatible with existing SfM processes, and its adaptive parameters can be adapted to various types of scenario data. It has the advantages of strong stability, low resource consumption, and good generalization. It can be widely used in practical engineering scenarios such as robot mapping, autonomous driving localization, and 3D content generation, and has outstanding practical value and promotion prospects.

[0137] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0138] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0139] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0140] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0141] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0143] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random-access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0144] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0145] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0147] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A hierarchical 3D modeling method using a scene skeleton as anchor points, characterized in that, Includes the following steps: S1. A weighted scene graph based on the input, using top- k The adjacency filtering strategy constructs a local subgraph, and the connected components on the local subgraph are calculated to obtain local communities; S2. Based on local communities, bridge edges are determined and island-level graphs are constructed. On the island-level graphs, skeleton subgraphs are constructed by merging disjoint spanning trees with the maximum weight edges. The bridge edges in the skeleton subgraphs are selectively verified using a dual-view deambiguer. S3. Take the images that have not entered the skeleton subgraph as leaf nodes and attach the leaf nodes to the skeleton nodes in the corresponding local community. S4. Reconstruct the scene skeleton based on the skeleton subgraph, and gradually absorb the leaf nodes through registration, triangulation and local binding adjustment to complete the complete 3D reconstruction.

2. The hierarchical 3D modeling method using a scene skeleton as anchor points according to claim 1, characterized in that, In step S1, the top- k The adjacency filtering strategy constructs a local subgraph, specifically: for each image, only the strongest edges sorted by edge weight are retained. k The edges corresponding to each of the neighbors, where... k The value is 1; the edge weight is the number of interior points of the image pair after geometric verification.

3. The hierarchical 3D modeling method using a scene skeleton as anchor points according to claim 1, characterized in that, Step S2 is as follows: The edges connecting different local communities are defined as bridging edges. An island-level graph is constructed with local communities as nodes and bridging edges as edges. The edge weights of the island-level graph inherit the number of interior nodes of the original edges. Select on the island-level map K A spanning tree with no overlap between any two edges is formed, and the set of edges with the largest total weight is retained to form the backbone. Traverse the bridge edges in descending order of edge weight, and only add bridge edges that satisfy the constraints of different connected components, island pair diversity, and dual-view deambiguer discrimination score not lower than the threshold to the spanning tree. By adding high-weight edges within a local community, the skeleton nodes within the local community are connected into a single connected subgraph, resulting in the skeleton subgraph.

4. The hierarchical 3D modeling method using a scene skeleton as an anchor point according to claim 3, characterized in that, In step S202, the number of spanning trees K According to the formula K =max(1,round( n / 300)) is determined, where, n This represents the total number of input images.

5. The hierarchical 3D modeling method using a scene skeleton as an anchor point according to claim 3, characterized in that, In step S203, the dual-view deambiguer is Doppelgangers++, and the discrimination process is implemented using a greedy Kruskal minimum spanning tree construction method.

6. The hierarchical 3D modeling method using a scene skeleton as anchor points according to claim 1, characterized in that, Step S3 is as follows: Within the local community to which a leaf node belongs, high-weight neighbors are selected for the leaf node and connected to the skeleton node first to form leaf attachment edges; the hierarchical reconstruction graph is composed of the edges of the skeleton subgraph and the leaf attachment edges.

7. The hierarchical 3D modeling method using a scene skeleton as anchor points according to claim 1, characterized in that, Step S4 is as follows: The incremental motion recovery structure solver is restricted to run on the skeleton subgraph to recover the global scene skeleton reconstruction results; Based on 2D The 3D correspondence and PnP solver are registered with leaf images, and local binding adjustments are performed on newly added cameras and 3D points. Duplicate registration triangulation The local optimization process involves performing global binding adjustments after all images have been absorbed to obtain the final reconstruction result.

8. The hierarchical 3D modeling method using a scene skeleton as anchor points according to claim 1, characterized in that, The weighted scene graph is constructed using standard feature extraction, matching, and geometric verification processes. The scene graph is represented as follows: G = ( V , E ),in, V Given a set of input images, E Image edges that are preserved through geometric verification.

9. A hierarchical 3D modeling system using a scene skeleton as anchor points, characterized in that, include: A partitioning module is used for weighted scene graphs based on the input, employing top-... k The adjacency filtering strategy constructs a local subgraph, and the connected components on the local subgraph are calculated to obtain local communities; The skeleton module, connected to the partitioning module, is used to determine bridging edges based on local communities and construct an island-level graph. On the island-level graph, a skeleton subgraph is constructed by merging the maximum weight edge non-intersecting spanning trees. The bridging edges in the skeleton subgraph are selectively verified using a dual-view deambiguer. The node module, connected to the skeleton module, is used to take images that have not entered the skeleton subgraph as leaf nodes and attach the leaf nodes to the skeleton nodes in the corresponding local community. The reconstruction module, connected to the node module, is used to reconstruct the scene skeleton based on the skeleton subgraph. It gradually absorbs leaf nodes through registration, triangulation, and local binding adjustments to complete the full 3D reconstruction.

10. The hierarchical 3D modeling system with scene skeleton as anchor point according to claim 9, characterized in that, The skeleton module includes: The island-level graph construction submodule is used to determine bridging edges and construct the island-level graph; The spanning tree merging submodule is used to select on an island-level graph. K The main skeleton is obtained by merging two non-overlapping spanning trees. The selective verification submodule is used to discriminate and verify bridge edges using a dual-view deambiguer; The island connectivity completion submodule is used to supplement high-weight edges within a local community, connecting skeleton nodes into a single connected subgraph.

11. The hierarchical 3D modeling system with scene skeleton as anchor point according to claim 9, characterized in that, The reconstruction module includes: The scene skeleton reconstruction submodule is used to restore the global scene skeleton on the skeleton subgraph; The leaf absorption submodule is used to register leaf images through the PnP solver and perform local triangulation and local bundling adjustments. The global optimization submodule is used to perform global bundled adjustments on all reconstructed models.

12. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform the method of any one of claims 1 to 8.

13. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including steps for performing the method of any one of claims 1 to 8.

14. A chip, characterized in that, A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-7.

15. An electronic device, characterized in that, Includes the chip as described in claim 14.

Images