Three-dimensional Mesh model hole repairing system and method considering curvature characteristics of scene structure

Abstract

The invention relates to a three-dimensional Mesh model hole repairing system and method considering curvature characteristics of a scene structure, and belongs to the technical field of three-dimensional modeling and graphic processing. Comprising the following steps: S1, identifying boundary holes in a three-dimensional Mesh model, and classifying the boundary holes according to form and location features; s2, constructing a hierarchical structure of the boundary holes based on a topological relation of the block grids; s3, repairing boundary holes layer by layer based on the hierarchical structure, wherein the boundary holes are converted into closed holes, and curved surfaces are fitted; and S4, smoothing the triangulation network of the repairing area and the adjacent boundary of the original model, so that the triangulation network is naturally transited. According to the method, each block grid serves as a basic repairing unit of the boundary holes, the hierarchical relation of the boundary holes is established, and the holes are repaired layer by layer from outside to inside, so that the hole repairing efficiency is improved, the applicability of the method is enhanced, and the repairing precision is ensured.

Description

Technical Field

[0001] This invention relates to a system and method for repairing holes in a 3D Mesh model that takes into account the curvature characteristics of the scene structure, belonging to the field of 3D modeling and graphics processing technology. Background Technology

[0002] Repairing holes in 3D mesh models is a core step in oblique photogrammetry-based 3D reconstruction, directly impacting the model's integrity, geometric accuracy, and engineering application value. Oblique photogrammetry generates point clouds through dense matching of multi-view images, which are then triangulated to construct mesh models. However, large-scale reconstruction requires segmented processing, and the boundaries of these segments are prone to "boundary holes"—missing triangular areas on the model surface—due to data gaps, weak texture areas (water bodies, smooth surfaces), or limitations of the shooting angle. These holes manifest as discontinuities or voids at the joints between segments, and in severe cases, can lead to topological breaks and elevation jumps, failing to meet the high-precision requirements of urban planning, surveying engineering, and other similar scenarios.

[0003] Current methods for repairing boundary holes are mainly divided into two categories: Traditional manual repair: While it can ensure accuracy by supplementing aerial images or manually editing point clouds in hole areas, it increases the workload in the field by 30%-50%, relies on human experience, is extremely inefficient, and cannot meet the needs of large-scale automated modeling.

[0004] Automated repair methods: Voxel method: Converts point cloud into a symbolic distance field and fills holes through volume diffusion, but requires processing massive voxel data, consumes a lot of memory, and is only suitable for small-scale models.

[0005] Surface fitting method: Fits the minimum area surface based on the hole boundary information. It is suitable for closed holes, but it has weak ability to handle boundary holes (especially open boundaries of block edges), and the repair success rate is <60%.

[0006] Aggregation repair method: As disclosed in the prior art: Han, J., & Shen, S. (2019). Scalable point cloud meshing for image-based large-scale 3D modeling. Visual Computing for Industry, Biomedicine, and Art, 2(1), 1-9; The disclosed content is as follows: Step 1: Based on the large-scale point cloud data obtained from the oblique image, the point cloud data is divided into blocks using a regular grid, and an octree index is established to uniformly manage all the block grids, such as Figure 1 (a) Figure 1 (b) and Figure 1 As shown in (c); Step 2: Using each segmented grid as a processing unit, the terrain surface within the block is reconstructed based on the point cloud within the block using the global optimal triangulation construction algorithm. During this process, the boundary lines do not participate in the mesh construction, but are only used as the range lines for triangulation reconstruction. Step 3: Complete the surface reconstruction within each segmented grid in sequence; Step 4: Merge the triangular mesh surfaces of each block unit based on the octree index. This can restore the boundary holes caused by the block division to closed holes, such as... Figure 1 As shown in (d); Step 5: Use the half-side data structure to identify whether there are holes in the block grid. If so, use the fast and robust minimum area method to reconstruct and subdivide the triangular mesh in the closed hole, thereby achieving hole repair.

[0007] Converting boundary holes into closed holes through block data aggregation is a relatively mature automated solution, but its core reliance on "global data fusion" has revealed significant shortcomings in practical applications.

[0008] Existing automated methods for large-scale oblique photogrammetry 3D reconstruction suffer from the following significant problems: Efficiency and memory bottlenecks: To aggregate boundary holes, multiple complete mesh data blocks (millions to hundreds of millions of triangular facets) need to be loaded, leading to a surge in computation time and high memory consumption; when the area is >2km 2 When the amount of data exceeds the single machine's memory limit, the program crashes and cannot be repaired.

[0009] Applicability limitations: Insufficient type coverage: It can only handle internal boundary holes (Type I) that form closed areas with adjacent blocks, and cannot repair external boundary holes (Type II) located at the edge of the model without adjacent blocks. In actual engineering, Type II holes account for 15%-20% (such as water bodies at the edge of cities and the edge of mountainous areas).

[0010] Poor morphological adaptability: For complex morphological holes such as hollow (without adjacent curved surfaces) and island (surrounding isolated curved surfaces), the aggregated structure easily forms a "pseudo-closed boundary," resulting in a repair success rate of <30%; area ≥1000m² 2 Due to weak boundary constraints, the surface fitting accuracy of the holes decreased by more than 40%.

[0011] Precision defects: After aggregation, the Euclidean distance between the center region of the hole and the boundary is large (>50m), and the elevation constraint of the boundary node on the center is weakened, causing the curved surface to deviate from the real terrain features. Summary of the Invention

[0012] To address the aforementioned issues, this invention aims to provide a three-dimensional mesh model hole repair system and method that takes into account the curvature characteristics of the scene structure, thereby solving the problems of low efficiency, poor applicability, and insufficient accuracy in boundary hole repair during large-scale oblique photogrammetry three-dimensional reconstruction.

[0013] The method for repairing holes in a 3D mesh model that takes into account the curvature characteristics of the scene structure, as described in this invention, includes the following steps: S1: Identify boundary holes in the 3D Mesh model and classify the boundary holes according to their morphological and locational characteristics; S2: Construct the hierarchical structure of the boundary holes based on the topological relationship of the segmented grid; S3: Repair the boundary holes layer by layer based on the hierarchical structure, including converting the boundary holes into closed holes and fitting a curved surface; S4: Smooth the triangular mesh at the boundary between the repaired area and the original model to make the triangular mesh transition naturally.

[0014] Preferably, in step S1, the identification of the boundary hole is achieved through a half-side data structure: traversing the triangular mesh edges of the Mesh model, if an edge is associated with only one triangle, it is determined to be a boundary edge; tracing the boundary edge along a preset direction to form a closed loop, if the loop contains the boundary of the segmented grid, it is determined to be a boundary hole.

[0015] Preferably, in step S1, the classification includes: Based on location characteristics, they are divided into internal boundary holes and external boundary holes; Based on morphological characteristics, they are divided into at least four subcategories, including hollow type, semi-enclosed type, through type and island type.

[0016] Preferably, in step S2, the construction of the hierarchical structure includes the following steps: S21: Index the segmented grid and cluster the boundary holes with topological relationships into at least one cluster; S22: Based on the spatial location of the boundary holes in the cluster, the structure is divided into a hierarchical structure from the outer layer to the inner layer.

[0017] Preferably, in step S21, the determination condition for the topological association is: the grid blocks containing the two boundary holes share a boundary, and the shortest Euclidean distance between the hole boundaries is less than a preset threshold.

[0018] Preferably, in step S3, the process of converting the boundary hole into a closed hole includes topological closure processing, which is implemented by constructing a buffer based on the boundary features of the boundary hole, selecting network nodes on the boundary of the buffer, and forming a topologically closed polygonal boundary.

[0019] Preferably, in step S3, the fitted surface includes elevation projection transformation, specifically: projecting boundary nodes onto the grid boundary to generate new nodes, calculating the average centroid elevation of the triangular facets associated with the boundary nodes, and assigning the average value to the corresponding new nodes.

[0020] Preferably, in step S3, the surface within the closed hole is fitted to the fitted surface to achieve hole repair, specifically including the following steps: S31: Hole triangulation: After the original mesh is topologically closed, closed holes are generated. The holes are triangulated using the maximum dihedral angle and minimum area criteria. S32: Mesh Subdivision: The triangulation mesh is subdivided by adding centroids. The specific subdivision steps are as follows: S321: Calculate each vertex of the hole boundary. Average length of connecting edges of associated first-order neighborhood vertices ; S322: Calculate the newly added triangular mesh center of gravity and length factor ; S323: (1) In the formula, The length of the edge connecting the vertex. This is the density control factor. The larger the density factor, the denser the subdivision mesh. (Default) If the conditions of formula (1) are met simultaneously, then for the triangular mesh... If the subdivision is not performed, then the subdivided triangular mesh is optimized using the Delaunay criterion. If the Delaunay criterion is not met, then edge swapping is performed. Finally, the above steps are repeated until no new meshes are added and no mesh edge swapping is performed. S33: Grid Adjustment: Based on the connection relationship of the newly added and repaired grid, inner rings are extracted. The outermost ring is adjusted first, then the next outermost ring, and so on, layer by layer. The position adjustment is performed using a distance-inverse weighted average. The first-order and second-order neighbors of the newly added nodes in the outermost layer are calculated; the weight of each neighboring node is calculated; and the position is adjusted layer by layer from the outside to the inside using a weighted average algorithm. The adjustment function is as follows: (2) In the formula: , For adding nodes The adjacent nodes; S33: Grid smoothing: Laplace smoothing is used, with the following formula: (3) In the formula, For cotangent weight, They are the edges The angle values ​​of the two corresponding adjacent triangles, Represents vertices The deviation from the adjacent triangular mesh is called the Laplace harmonic umbrella operator, and the second-order umbrella operator can be obtained through recursion; (4) In the formula, This means that the deviation between a vertex and a triangular mesh is equal to the average deviation of adjacent triangular meshes, which is equivalent to reducing mesh optimization to linear equations and completing the final smoothing process.

[0021] Preferably, in step S4, the smoothing process of the triangular mesh of the boundary between the repair area and the original model specifically includes the following: using the Laplace smoothing method to smooth the elevation of the shared boundary, and assigning the elevation of each node to the average value of the node elevation within a certain buffer range; For each point to be repaired, let its original elevation be... Define its updated elevation value as : (5) In the formula, To achieve smooth sparseness, , The definition is as follows: (6) In the formula, For the point of Neighboring points, The number of neighboring points.

[0022] The present invention provides a hole repair system for a 3D mesh model that takes into account the curvature characteristics of the scene structure, comprising: Hole recognition and classification module: used to perform step S1; Hierarchical structure construction module: used to perform step S2; Layer-by-layer repair module: used to perform step S3, including topology closure unit, elevation projection unit and surface fitting unit; Boundary smoothing module: used to perform step S4, and to achieve smoothing using formulas (5) and (6).

[0023] Compared with existing technologies, the present invention provides a three-dimensional mesh model hole repair system and method that takes into account the curvature characteristics of scene structures, and has the following advantages: 1. Overcome the bottleneck in efficiency for large-scale repairs By adopting a segmented grid independent processing strategy to avoid global data aggregation, and combined with hierarchical layer-by-layer repair logic, the computational complexity and memory consumption are significantly reduced. It can stably support the repair of boundary holes in ultra-large-scale areas, breaking through the scale limitations of traditional methods.

[0024] 2. Comprehensive coverage of complex hole types By classifying holes by location and morphology (inner / outer boundary holes, hollow / semi-enclosed / through-type / island type), and combining hierarchical clustering to construct the repair relationship between the outer layer and the inner layer, the method can effectively repair holes with complex shapes and outer boundary holes, greatly expanding its applicability.

[0025] 3. Improve the accuracy and naturalness of surface fitting. An innovative collaborative optimization mechanism combining topological closure and elevation projection is used to assign reasonable topological and elevation constraints to the hole boundaries. A curvature-aware Laplacian smoothing algorithm is adopted to maintain the curvature consistency between the repaired area and the original model, ensuring a natural transition of the surface and meeting the requirements of high-precision modeling.

[0026] (1) Regarding the repair success rate: For closed holes with a large proportion, the repair results of the method of the present invention are consistent with those of the existing methods; for Class I boundary holes, the repair success rate of the method of the present invention is 12.4% higher than that of the mature existing methods. In addition, the method of the present invention makes up for the shortcomings of the existing methods in that they cannot complete the repair of Class II boundary holes, and has better applicability.

[0027] (2) In terms of repair efficiency: Existing methods can only complete repairs in small areas (≤2km) 2 Repairing holes, when the experimental area is large (>2km) 2 This method, however, can cause computer crashes due to the large amount of data to be processed, making it impossible to repair the holes. Within the repairable experimental range, the time consumption of existing methods is approximately 4-5 times that of the method of this invention. Furthermore, in terms of memory consumption, within the repairable experimental range, the memory consumption of existing methods is approximately 7-21 times that of the method of this invention.

[0028] (3) In terms of repair accuracy: Compared with existing methods, the method of the present invention improves the average elevation difference by 37.5% and the standard deviation of elevation difference by 38.9%, with higher overall elevation accuracy and smaller error.

[0029] 4. Promote the implementation of engineering applications By improving efficiency and expanding applicability, reducing manual intervention and field costs, and shortening the modeling cycle, it can be widely applied to the 3D reconstruction of complex terrain and large-scale areas, promoting the industrial application of oblique photogrammetry technology in fields such as digital twins and intelligent surveying and mapping. Attached Figure Description

[0030] Figure 1 The diagram illustrates the principle of boundary hole repair using existing methods. In the diagram, (a) represents the original closed hole, (b) represents the block grid, (c) represents the unclosed hole generated by the block grid, and (d) represents the closed hole generated by the aggregation of the unclosed hole. Figure 2 The diagram shows a hole, where (a) represents a half-side data structure and (b) represents an open hole. Figure 3 This is a schematic diagram of the coarse classification of unclosed holes in this invention. In the figure, (a) represents an internal unclosed hole, and (b) represents an external unclosed hole. Figure 4 This is a schematic diagram of the classification of non-closed holes in this invention. In the diagram, (a) represents hollow type, (b) represents semi-enclosed type, (c) represents through type, and (d) represents island type. Figure 5 This is a schematic diagram of the hierarchical relationship in the present invention. In the figure, (a) represents an unclosed hole and (b) represents a segmented grid. Figure 6 This is a schematic diagram of topological closure in the present invention. In the figure, (a) represents type 1, (b) represents type 2, and (c) represents type 3. Figure 7 This is a schematic diagram of the topological closure of the non-closed hole in this invention. In the figure, (a) represents the hollow type, (b) represents the semi-enclosed type, (c) represents the through type, and (d) represents the island type. Figure 8 This is a schematic diagram of elevation propagation in this invention. In the figure, (a) represents the identification of each boundary node, and (b) represents the elevation transfer assignment. Figure 9 This is a schematic diagram of hole repair in the present invention. In the figure, (a) original mesh, (b) hole topological closure, (c) hole triangulation, (d) mesh subdivision, (e) adjacent point extraction, and (f) height adjustment. Figure 10 This is a schematic diagram of the cotangent weight angle in this invention; Figure 11 This is a schematic diagram showing the common boundary and elevation difference between adjacent holes in this invention; Figure 12 The figures show the hole repair results of the two methods in this invention; in the figures, (a) represents the Han method and (b) represents the method of this invention. Figure 13 The diagram shows the distribution of holes after repair in this invention; in the diagram, (a) represents the distribution of holes, and (b) represents the repair of holes. Figure 14 This is a histogram showing the repair time statistics in this invention; Figure 15 This is a histogram showing the memory consumption statistics in this invention; Figure 16 The figure shows the experimental data results of this invention; in the figure, (a) represents the original grid, (b) represents artificially dug holes, and (c) represents hole segmentation; Figure 17 The figure shows the experimental data results of the Han method in the prior art; in the figure, (a) represents hole aggregation, (b) represents hole repair, and (c) represents a local detail of the hole. Figure 18 The figure shows the experimental data results of the method of the present invention; in the figure, (a) represents hole aggregation, (b) represents hole repair, and (c) represents a local detail of the hole. Figure 19 This is a comparison chart of repair accuracy. Detailed Implementation

[0031] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0032] Example 1: This embodiment discloses a method for repairing holes in a 3D mesh model that takes into account the curvature characteristics of the scene structure, including the following steps: S1: Identify boundary holes in the 3D Mesh model and classify the boundary holes according to their morphological and locational characteristics; Specifically, this embodiment uses a half-edge data structure for boundary hole identification. The half-edge data structure is used to organize the triangular faces within the block to quickly identify adjacent triangles associated with a particular edge, such as... Figure 2 As shown in (a), edge e 0,1 There are two halves of h facing opposite directions. 0,1 and h 1,0 This indicates that it is associated with two triangles; if a side is associated with only one triangle, it is defined as a boundary edge. Boundary edge tracing is performed along a fixed direction (clockwise or counterclockwise). If the resulting loop contains a segmented grid boundary, then the hole is a boundary hole. Figure 2 As shown in (b).

[0033] Based on the location of boundary holes in the reconstruction area, boundary holes are first divided into two categories: Category I—internal boundary holes, that is, the boundary of the grid blocks contained in the boundary hole is not the boundary of the reconstruction area, such as... Figure 3 As shown in (a); Type II—external boundary holes, that is, the boundary holes contain grid blocks whose boundaries are the boundaries of the reconstructed region, such as... Figure 3 As shown in (b).

[0034] Furthermore, based on the morphological characteristics of the boundary holes, they are divided into four subcategories: Type A: Hollow type: Boundary holes fill the entire block grid area, meaning there are no adjacent reconstructed surfaces such as... Figure 4 As shown in (a); Type B: Semi-enclosed type: Boundary holes are located on one side of the segmented grid area, and the other side is a reconstructed surface, such as... Figure 4 As shown in (b); Type C: Through-type: Boundary holes are located in the middle of the segmented grid region, with reconstructed surfaces on both sides, such as... Figure 4 As shown in (c); Type D: Island type: The boundary hole is located in the middle of the segmented grid region, with reconstructed surfaces on both sides, and one (or both) of them are island-shaped, such as... Figure 4 As shown in (d).

[0035] S2: Construct the hierarchical structure of the boundary holes based on the topological relationship of the segmented grid; Specifically, firstly, a grid index is established to organize the original block grid. For two boundary holes, if their block units are adjacent to each other and there is a shared grid boundary between the holes, they are clustered and identified, but no merging operation is performed. The above process is iterated until all boundary holes participate in the calculation, and the cluster in which each boundary hole belongs is obtained. Secondly, the boundary holes in each cluster are divided into layers, as follows: (1) The boundary hole located on the outermost side of the cluster is the first layer, which can be determined by identifying the boundary edge in the initial triangular mesh, that is, the boundary hole where the boundary edge is located is in the first layer; (2) The hole that shares a boundary with the first layer boundary hole but does not belong to the first layer is the second layer hole; (3) and so on, to obtain the nth layer hole, thereby establishing the hierarchical relationship between boundary holes. Figure 5 As shown, for Figure 5 The hierarchical relationship between the boundary hole clusters in (a) is as follows: Figure 5 As shown in (b).

[0036] S3: Repair the boundary holes layer by layer based on the hierarchical structure, including converting the boundary holes into closed holes and fitting a curved surface; Specifically, after establishing the hierarchical relationship of boundary holes, repair is carried out layer by layer, starting from the outermost hole (layer 1) and proceeding "from the outside in". There are two key issues in the repair process: (1) how to convert boundary holes into closed holes, that is, how to achieve boundary topological closure; (2) how to reasonably fit the surface of the holes within the block, that is, how to assign appropriate elevation values ​​to the network nodes near the grid boundary. To this end, a unified topological closure principle is established to perform multi-type boundary hole closure processing, and elevation value projection transformation is performed to complete the layer-by-layer repair of boundary holes.

[0037] First, the boundary of any hole is divided into two parts: the curved surface boundary and the grid boundary. The former is the boundary edge of the original triangular curved surface that constitutes the hole, and the latter is the block grid boundary that constitutes the hole. Although the grid boundary is part of the boundary hole, the connection between it and the curved surface boundary is not topologically closed. Therefore, it is difficult to convert it into a closed hole for repair. This invention takes topological constraints into account for boundary closure. It analyzes the boundary characteristics of four types of boundary holes. Their topological closure can be summarized into the following three cases: (1) The first and last two points of the curved surface boundary are located on both sides of the grid boundary. Then, with the grid boundary as the center, buffer zones are established on both sides of the grid boundary with the distance from the first and last two points to the grid boundary as constraints. Then, the mesh nodes are selected on the buffer zone boundary to form a topologically closed triangular mesh, such as Figure 6 As shown in (a); (2) if one endpoint of the surface boundary edge is located on the grid boundary and the two boundary edge nodes associated with that node are located on both sides of the grid boundary, then with the grid boundary as the center, and with the distances from the two associated boundary edge nodes to the grid boundary as constraints, buffer zones are established on both sides of the grid boundary, and then network nodes are selected on the buffer zone boundary to form a topologically closed triangular network, such as Figure 6 As shown in (b); (3) if an endpoint of a curved surface boundary edge is located on the grid boundary and the two boundary edge nodes associated with that node are located inside the grid boundary, then a buffer zone is established outside the grid boundary with the grid boundary as the center and the average value of the two boundary edges associated with that node as the distance constraint. Then, network nodes are selected on the buffer zone boundary to form a topologically closed triangular network, such as Figure 6 As shown in (c).

[0038] Based on the above topology closure rules, different closure strategies are used for topology closure of different types of unclosed holes. For type A – “hollow” unclosed holes, buffer zones are constructed in the directions inside and outside the grid boundary based on the segmented nodes, and then topology closure is performed, such as… Figure 7 As shown in (a). For type B – “semi-enclosed” unclosed holes, buffer zones are constructed in the directions inside and outside the grid boundary based on the hole boundary and the dividing nodes, and then topological closure is performed, as follows. Figure 7 As shown in (b). For Class C – “through-type” unclosed holes, buffer zones are constructed in the directions inside and outside the grid boundary based on the hole boundary, and then topological closure is performed, as follows. Figure 7 As shown in (c). For type D – “island-type” unclosed holes, a buffer zone is constructed in the inward and outward directions at the shortest point between the island boundary and the hole boundary, and then topological closure is performed, as shown in Figure (c). Figure 7 As shown in (d).

[0039] To ensure the accuracy of the fitted surface for the holes within the block, appropriate elevation values ​​need to be assigned to the grid nodes near the grid boundary. Therefore, an elevation value projection transformation is performed. First, the boundary nodes are projected onto the grid boundary, and the projected points are used as new nodes on the grid boundary to ensure that the new triangular faces have similar compactness to the original triangular faces. Figure 8 As shown in (a); next, identify the boundary triangular facets associated with each boundary node, and calculate the average elevation of the centroid of each boundary triangular facet; finally, assign the elevation value associated with each boundary node to its corresponding newly added node, as shown in (a). Figure 8 As shown in (b). To improve the accuracy of elevation assignment, this invention first uses the four corner points of the block grid as dividing points to divide the block grid boundary into four parts, and then uses the above algorithm to obtain the new nodes of each part boundary and their corresponding elevation values.

[0040] In the process of repairing boundary holes, the elevation of the grid boundary in each block unit is first assigned in the first layer, and then this value is passed to the second layer. The propagation process is consistent with the intra-block elevation propagation process. Based on the above algorithm, the boundary hole can be converted into a topologically closed hole, and the grid boundary has suitable nodes for triangulation. Therefore, the surface inside the closed hole can be fitted according to the minimum and maximum dihedral angle and area method, thereby realizing the hole repair.

[0041] The specific steps are as follows: (1) Steps for triangulation of holes: as follows Figure 9 As shown in (a), the original mesh generates closed holes after topological closure, as shown in (a). Figure 9 As shown in (b), the hole is triangulated using the "maximum dihedral angle and minimum area" criterion. The specific triangulation rules are: (1) Prioritize adding triangular faces that form the maximum dihedral angle with the neighboring triangulation network; (2) Then consider adding triangular faces with the minimum area. Iterate through the holes to triangulate all vertices, as shown in (b). Figure 9 As shown in (c).

[0042] (2) Mesh subdivision steps: Triangulation meshes are subdivided by adding centroids, such as... Figure 9 As shown in (d), the specific subdivision steps of the grid are as follows: First, calculate each vertex of the hole boundary. Average length of connecting edges of associated first-order neighborhood vertices Then calculate the newly added triangular mesh. center of gravity and length factor If the conditions of formula (1) are met simultaneously, then for the triangular mesh... If the subdivision is not performed, then the subdivided triangular mesh is optimized using the Delaunay criterion. If the Delaunay criterion is not met, then edge swapping is performed. Finally, the above steps are repeated until no new meshes are added and no mesh edge swapping is performed.

[0043] (1) In the formula, The length of the edge connecting the vertex. This is the density control factor. The larger the density factor, the denser the subdivision mesh. This invention assumes... .

[0044] (3) Grid adjustment steps: The newly added repair grid adopts an adjustment order from the outside to the inside. Based on the connection relationship of the newly added repair grid, inner rings are extracted. First, the outermost ring is adjusted, then the next outermost ring is adjusted, and so on, layer by layer. The position adjustment is performed using the inverse distance weighted average. The farther away from the new node, the smaller the impact on the position of the new node. First, calculate the first-order neighbors (blue nodes) and second-order neighbors (green nodes) of the outermost new node, such as... Figure 9 As shown in (e); secondly, calculate the weight of each adjacent node; finally, use a weighted average algorithm to adjust the position layer by layer from the outside to the inside, first adjusting the dark gray nodes, then adjusting the light gray nodes, as shown in (e). Figure 9 As shown in (f). The adjustment function is as follows.

[0045] (2) In the formula , For adding nodes Adjacent nodes, such as Figure 9 The node indicated by the yellow arrow in (f).

[0046] (4) Grid smoothing step: In order to make the new grid smoother, the grid needs to be smoothed. Curvature-based global smoothing can better maintain the original approximate shape of the model. The existing popular grid smoothing method is to take the weighted center of the ring neighborhood of each point as the position of the new vertex, which is defined as the weighted umbrella operator, which is actually solving the Laplacian matrix.

[0047]

[0048] In the formula, It is the vertex The first-order neighborhood vertex; It is the weight value of the edge.

[0049] The difference between curvature-based smoothing and the Laplace smoothing mentioned above is that curvature-based smoothing works along the vertex. Use the normal vector to move the vertex The conventional Laplace smoothing, on the other hand, is to smooth the vertex... Move to a position similar to the center of gravity. It can be deduced that by selecting appropriate Laplace weights, the calculation formula is shown in (4), and Laplace smoothing can achieve the same effect as curvature-based smoothing.

[0050] (3) In the formula, For cotangent weight, They are the edges The angle values ​​of the two corresponding adjacent triangles, such as Figure 10 As shown. Represents vertices The deviation from the adjacent triangular mesh is called the Laplace harmonic umbrella operator. The second-order umbrella operator can be obtained through recursion.

[0051] (4) In the formula, This means that the deviation between a vertex and a triangular mesh is equal to the average deviation of adjacent triangular meshes, which is equivalent to reducing mesh optimization to linear equations and completing the final smoothing process.

[0052] S4: Smooth the triangular mesh at the boundary between the repaired area and the original model to make the triangular mesh transition naturally.

[0053] Specifically, after layer-by-layer processing, the repair of all boundary holes within the segmented units is completed. However, because the processing is done layer by layer, such as Figure 11 As shown, there will be an elevation difference at the common boundary of two adjacent holes in the same layer. Therefore, it is necessary to smooth the common boundary of adjacent holes in the same layer.

[0054] Laplacian smoothing is widely used for surface smoothing. It can homogenize the mesh while preserving the local geometric details of the surface. It is also simple to compute and has a short computation time. Therefore, this invention uses the Laplacian smoothing method to smooth the shared boundary elevation. The principle is to assign the elevation of each node to the average value of the node elevations within a certain buffer range.

[0055] For each point to be repaired, let its original elevation be... Define its updated elevation value as : (5) In the formula, To achieve smooth sparseness, , The definition is as follows: (6) In the formula, For the point of Neighboring points, The number of neighboring points.

[0056] Experiments and Analysis The experimental area was a portion of the built-up area of ​​a city, measuring 5.2km × 7.8km. A 5-lens UltraCam OspreyPrima (UCOp) camera (1 front-view + 4 side-view) flew along 29 flight paths, acquiring 11,795 images, totaling 2.08TB of data. The reconstructed area is approximately 28km. 2 The reconstructed area was divided into 449 sub-regions with a grid size of 250m × 250m. The operating environment was a standard personal computer with a Windows 10 64-bit operating system, an Intel Xeon(R) E3-1535M CPU with a clock speed of 3.10GHz, and 64GB of RAM. The method of this invention was compared and analyzed with the existing method of Han et al. (2019) (hereinafter referred to as the Han method) to verify the effectiveness and superiority of the present invention. The experiment consisted of three parts: comparison and verification of repair success rate, comparison and verification of repair efficiency, and comparison and verification of repair results.

[0057] (1) Comparison of repair success rates in localized areas of the experimental zone Select a location 2km in the middle of the reconstruction area (with abundant water). 2 A comparative experiment was conducted on the success rate of repairing unclosed holes in the region. The repair results of the two methods are as follows: Figure 12 (a) and Figure 12 As shown in (b), the area of ​​the unclosed holes within the experimental region ranges from [1242.99m²]. 2 13703.97m 2 The distribution and repair status of closed holes, Class I and Class II open holes are shown in Tables 1 and 2.

[0058] Table 1. Distribution and repair status of closed holes and Class I open holes.

[0059] Table 2. Distribution and Repair Status of Type II Unclosed Holes

[0060] Table 1 shows that there are 16 closed holes located inside the segmented grid within the experimental area. Since the method of this invention uses the same minimum area method as the Han method in the hole repair process, both can repair closed holes with a repair rate of 100%. For type I unclosed holes, Type-B holes are the most numerous (17), while Type-C holes are the fewest (0). Overall, the repair success rate of the Han method is 50%, while the repair success rate of the method of this invention is 100%, indicating a higher success rate. The main difference between the two methods lies in the handling of Type-A and Type-B holes. Comparing the actual data, it was found that the two Type-A unclosed holes and eight Type-B unclosed holes that failed to be repaired by the Han method constituted a single unclosed hole with a very large area, leading to its repair failure.

[0061] As shown in Table 2, compared with type I unclosed holes, the number of type II unclosed holes is significantly less, with only two. For type II unclosed holes, the Han method cannot complete the repair because they cannot form a closed hole with adjacent holes or there are no adjacent holes around them. In contrast, the repair success rate of the method of this invention is 100%, indicating that the method of this invention has better applicability.

[0062] (2) Comparison of repair success rates in localized areas of the experimental zone Within a global area of ​​28km in the experimental zone 2 A comparative experiment was conducted on the success rate of repairing unclosed holes in the region. The holes and repair results are as follows: Figure 13 (a) and Figure 13 As shown in (b), the distribution and repair status of closed holes, Class I and Class II open holes are shown in Tables 3 and 4.

[0063] Table 3. Distribution and repair status of closed holes and Class I open holes.

[0064] Table 4. Distribution and Repair Status of Type II Unclosed Holes

[0065] Tables 3 and 4 show that there were 56 closed holes, 54 type I open holes, and 6 type II open holes in the experimental area. Due to the large experimental area, the Han method required loading all mesh data with holes for hole aggregation, resulting in memory overflow and abnormal program termination, thus failing to complete hole repair. In contrast, the method of this invention does not require hole aggregation; it loads mesh data with holes block by block and performs topological closure, thus enabling the repair of all holes.

[0066] Repair efficiency comparison and verification A 0.5km radius was selected within the reconstruction area. 2 1km 2 1.5km 2 2km 2 2.5km 2 and 28km 2 Six regions were used for boundary hole repair experiments, and the time and memory consumption of the Han method and the method of this invention were statistically analyzed and compared. The statistical time started from the completion time of intra-block triangular mesh reconstruction and ended with the completion time of boundary hole repair, mainly covering the processes of boundary hole identification, repair, and smoothing.

[0067] (1) Comparison of repair time Table 5 shows the repair time statistics for the two methods in experimental areas of different sizes, and the corresponding histograms are shown below. Figure 14 .

[0068] From Table 5 and Figure 14 It can be observed that: (1) as the experimental area increases, the time consumed by both methods increases, but overall, the time consumed by the method of this invention is lower than that of the Han method; (2) the Han method can only complete small-scale areas (≤2km). 2 Repairing holes, when the experimental area is large (>2km) 2 (3) Within the repairable experimental range, the Han method takes about 4-5 times longer than the method of this invention.

[0069] Table 5. Comparison of repair time for unclosed holes using the two methods in different experimental areas.

[0070] Note: The statistical time is in minutes.

[0071] Table 6. Comparison of memory repair methods for unclosed holes in different experimental areas using the two approaches.

[0072] Note: The statistical unit is GB. The statistical value of the method of this invention is the maximum memory consumption during the hole repair process.

[0073] (2) Comparison of memory consumption Table 6 shows the memory consumption statistics for the two methods within experimental regions of different sizes, and the corresponding histograms are shown below. Figure 15 .

[0074] From Table 6, Figure 16 (a) Figure 16 (b) and Figure 16From (c), it can be observed that: (1) as the experimental area increases, the memory consumption of both methods shows an increasing trend, but the memory consumption of the Han method increases significantly, while the memory consumption of the method of this invention increases slowly; (2) in a single-machine environment, when the experimental area is large (>2km), 2 (3) Within the repairable experimental range, the memory consumption of the Han method is about 7-21 times that of the method of the present invention.

[0075] Comparison and verification of repair results To verify the repair results and accuracy of the method of the present invention, a closed hole and an open hole were artificially dug in the area of ​​the intact triangular mesh surface within the experimental area, with an area of ​​76112.5 m². 2 The area is divided into four sub-regions. Closed holes are further divided into four internal open holes, and open holes are further divided into two external open holes. Figure 17 (a) Figure 17 (b) and Figure 17 As shown in (c).

[0076] Using the aforementioned hole data as input data, hole repair was performed using both the Han method and the method of this invention, and the repair results were compared and analyzed.

[0077] (1) Comparison of repair results from Figure 17 (a) Figure 17 (b) and Figure 17 As can be seen in (c), after the Han method aggregates the voids, the internally open voids are transformed into internally closed voids, which can be repaired; the externally open voids remain open voids and cannot be repaired. From Figure 18 (a) Figure 18 (b) and Figure 18 As can be seen in (c), after the hole is closed using the method of the present invention, both the internal and external unclosed holes are transformed into closed holes, enabling hole repair. Furthermore, regarding the repair results at the boundary of the closed hole, the hole boundary line is clearly visible in the Han method, while the method of the present invention allows for a natural transition.

[0078] (2) Comparison of repair accuracy The repair accuracy was evaluated by comparing the elevation values ​​of the repaired triangulated network nodes with those of the original nodes. In this experiment, the elevation range of the entire hole was 3.96m to 5.98m, with a difference of 2.02m. Thirty-four evenly distributed nodes were selected within the closed hole area, and the differences between the fitted elevation values ​​of the two methods and the original elevation values ​​at each node were statistically analyzed. The results are shown below. Figure 19 As shown.

[0079] Depend on Figure 19 It can be seen that the height difference of the mesh repaired by the method of this invention is smaller than that of the mesh repaired by the Han method. Compared with the original mesh, the average difference in elevation direction between the mesh repaired by the Han method and the original mesh is 16cm, and the maximum difference is 36cm; correspondingly, the average difference in elevation direction between the mesh repaired by the method of this invention and the original mesh is 10cm, and the maximum difference is 23cm. Further analysis of the standard deviation of the elevation difference shows that the standard deviation of the elevation difference for the Han method is 18cm, while the standard deviation for the elevation difference for the method of this invention is 11cm, indicating that the overall elevation accuracy of the nodes obtained by the method of this invention is higher and the error is smaller. Comparing with actual data, it was found that the closer the node is to the center of the hole, the greater its elevation difference with the original node. This is directly related to the process of hole triangulation and filling, and also indicates that the repair accuracy decreases as the size of the hole increases.

[0080] Example 2: The 3D Mesh model hole repair system that takes into account the curvature characteristics of scene structure as described in this invention includes: Hole recognition and classification module: Identifies boundary holes in the 3D Mesh model and classifies the boundary holes according to their shape and location characteristics; Hierarchical structure construction module: Constructs the hierarchical structure of the boundary holes based on the topological relationship of the segmented grid; Layer-by-layer repair module: Based on the hierarchical structure, repairs boundary holes layer by layer, including converting boundary holes into closed holes and fitting curved surfaces, including topological closure units, elevation projection units and curved surface fitting units; Boundary smoothing module: Smooths the triangular mesh of the boundary between the repaired area and the original model, so that the triangular mesh transitions naturally. The smoothing is achieved by formulas (5) and (6).

[0081] The system in this embodiment is based on the method in embodiment 1 and implements: (1) Boundary hole identification and classification: Boundary holes are identified based on the half-side data structure and classified into 2 major categories and 4 minor categories according to their shape and location characteristics; (2) Calculation of boundary hole hierarchical relationship: A block cell network index is established, and the boundary hole hierarchical relationship is calculated based on the adjacency relationship between the block units; (3) Boundary hole layer by layer propagation and repair: Boundary hole closure is performed based on the topological closure principle, and elevation value projection conversion is performed to complete the layer by layer repair of boundary holes; (4) Adjacent hole smoothing: Adjacent boundary smoothing is performed based on the Laplace algorithm to achieve natural transition of triangular networks between block units.

[0082] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for repairing holes in a 3D mesh model that takes into account the curvature characteristics of the scene structure, characterized in that, Includes the following steps: S1: Identify boundary holes in the 3D Mesh model and classify the boundary holes according to their morphological and locational characteristics; S2: Construct the hierarchical structure of the boundary holes based on the topological relationship of the segmented grid; S3: Repair the boundary holes layer by layer based on the hierarchical structure, including converting the boundary holes into closed holes and fitting a curved surface; S4: Smooth the triangular mesh at the boundary between the repaired area and the original model to make the triangular mesh transition naturally.

2. The method for repairing holes in a 3D mesh model that takes into account the curvature characteristics of the scene structure according to claim 1, characterized in that, In step S1, the identification of the boundary hole is achieved through a half-side data structure: traversing the triangular mesh edges of the Mesh model, if an edge is associated with only one triangle, it is determined to be a boundary edge; tracing the boundary edge along a preset direction to form a closed loop, if the loop contains the boundary of the segmented grid, it is determined to be a boundary hole.

3. The method for repairing holes in a 3D mesh model that takes into account the curvature characteristics of the scene structure according to claim 1, characterized in that, In step S1, the classification includes: Based on location characteristics, they are divided into internal boundary holes and external boundary holes; Based on morphological characteristics, they are divided into at least four subcategories, including hollow type, semi-enclosed type, through type and island type.

4. The method for repairing holes in a 3D mesh model that takes into account the curvature characteristics of the scene structure according to claim 1, characterized in that, In step S2, the construction of the hierarchical structure includes the following steps: S21: Index the segmented grid and cluster the boundary holes with topological relationships into at least one cluster; S22: Based on the spatial location of the boundary holes in the cluster, the structure is divided into a hierarchical structure from the outer layer to the inner layer.

5. The method for repairing holes in a 3D mesh model that takes into account the curvature characteristics of the scene structure according to claim 4, characterized in that, In step S21, the determination condition for the topological association is: the grid blocks containing the two boundary holes share a boundary, and the shortest Euclidean distance between the hole boundaries is less than a preset threshold.

6. The method for repairing holes in a 3D mesh model that takes into account the curvature characteristics of the scene structure according to claim 1, characterized in that, In step S3, the process of converting boundary holes into closed holes includes topological closure processing, which is achieved by constructing a buffer based on the boundary features of the boundary holes, selecting network nodes on the boundary of the buffer, and forming a topologically closed polygonal boundary.

7. The method for repairing holes in a 3D mesh model that takes into account the curvature characteristics of the scene structure according to claim 6, characterized in that, In step S3, the fitted surface includes elevation projection transformation, specifically: projecting boundary nodes onto the grid boundary to generate new nodes, calculating the average centroid elevation of the triangular facets associated with the boundary nodes, and assigning the average centroid elevation to the corresponding new nodes.

8. The method for repairing holes in a 3D mesh model that takes into account the curvature characteristics of the scene structure according to claim 1, characterized in that, In step S3, the fitted surface is used to fit the surface inside the closed hole, thereby achieving hole repair. Specifically, it includes the following steps: S31: Hole triangulation: After the original mesh is topologically closed, closed holes are generated. The holes are triangulated using the maximum dihedral angle and minimum area criteria. S32: Mesh Subdivision: The triangulation mesh is subdivided by adding centroids. The specific subdivision steps are as follows: S321: Calculate each vertex of the hole boundary. Average length of connecting edges of associated first-order neighborhood vertices ; S322: Calculate the newly added triangular mesh center of gravity and length factor ; S323： （1） in, Center of gravity To the top The vector value, The length of the edge connecting the vertex. This is the density control factor. The larger the density factor, the denser the subdivision mesh. (Default) If the conditions of formula (1) are met simultaneously, then for the triangular mesh... If the subdivision is not performed, then the subdivided triangular mesh is optimized using the Delaunay criterion. If the Delaunay criterion is not met, then edge swapping is performed. Finally, the above steps are repeated until no new meshes are added and no mesh edge swapping is performed. S33: Grid Adjustment: Based on the connection relationship of the newly added and repaired grid, inner rings are extracted. The outermost ring is adjusted first, then the next outermost ring, and so on, layer by layer. The position adjustment is performed using a distance-inverse weighted average. The first-order and second-order neighbors of the newly added nodes in the outermost layer are calculated; the weight of each neighboring node is calculated; and the position is adjusted layer by layer from the outside to the inside using a weighted average algorithm. The adjustment function is as follows: （2） In the formula: To add a new node, , For adding nodes The adjacent nodes; S33: Grid smoothing: Laplace smoothing is used, with the following formula: （3） In the formula, For cotangent weight, They are the edges The angle values ​​of the two corresponding adjacent triangles, Represents vertices The deviation from the adjacent triangular mesh is called the Laplace harmonic umbrella operator, and the second-order umbrella operator can be obtained through recursion; （4） In the formula, For cotangent weight, This means that the deviation between a vertex and a triangular mesh is equal to the average deviation of adjacent triangular meshes, which is equivalent to reducing mesh optimization to linear equations and completing the final smoothing process.

9. The method for repairing holes in a 3D Mesh model that takes into account the curvature characteristics of the scene structure according to claim 1, characterized in that, In step S4, the smoothing process of the triangular mesh of the boundary between the repair area and the original model specifically includes the following: using the Laplace smoothing method to smooth the elevation of the shared boundary, and assigning the elevation of each node to the average value of the node elevation within a certain buffer range. For each point to be repaired, let its original elevation be... Define its updated elevation value as : （5） In the formula, To achieve smooth sparseness, , The definition is as follows: （6） In the formula, For the point of Neighboring points, The number of neighboring points.

10. A hole repair system for a 3D mesh model that takes into account the curvature characteristics of the scene structure, based on the hole repair method for a 3D mesh model that takes into account the curvature characteristics of the scene structure as described in any one of claims 1-9, characterized in that, include: Hole recognition and classification module: used to perform step S1; Hierarchical structure construction module: used to perform step S2; Layer-by-layer repair module: used to perform step S3, including topology closure unit, elevation projection unit and surface fitting unit; Boundary smoothing module: used to perform step S4 to achieve triangulation smoothing.

Images