Three-dimensional mesh model hole repairing system and method considering scene structure curvature features

Abstract

The present application relates to a kind of three-dimensional Mesh model hole repair system and method considering scene structure curvature feature, belong to three-dimensional modeling and graphics processing technical field;Including the following steps: S1: identifying boundary hole in three-dimensional Mesh model, and according to shape and zone feature to the boundary hole classified;S2: based on the topological relationship of block grid, the hierarchical structure of the boundary hole is constructed;S3: based on the hierarchical structure, boundary hole is repaired layer by layer, including converting boundary hole into closed hole and fitting curved surface;S4: to the triangle net of repair area and original model adjoining boundary, it is carried out smoothing processing, so that triangle net naturally transitions.The present application takes each block grid as boundary hole basic repair unit, and establishes boundary hole hierarchical relationship, " from outside to inside", layer by layer repair, improves hole repair efficiency, enhances the applicability of method and guarantees repair precision.

Description

TECHNICAL FIELD

[0001] The present application relates to a three-dimensional Mesh model hole repairing system and method considering the curvature characteristics of scene structure, and belongs to the technical field of three-dimensional modeling and graphics processing. BACKGROUND

[0002] Three-dimensional Mesh model hole repairing is a core link of real scene three-dimensional reconstruction of oblique photography, and directly affects the integrity, geometric accuracy and engineering application value of the model. Oblique photography technology generates point cloud through multi-view image dense matching, and constructs Mesh model through triangulation, but large-scale reconstruction needs to be processed in blocks, and "boundary hole" may be generated at the block boundary due to data loss, weak texture area (water, smooth road surface) or shooting angle limitation, that is, the triangular missing area of the model surface, which is manifested as discontinuous surface or hole at the block joint, and in severe cases, the model topological fracture and height jump, which cannot meet the high-precision scene demand of city planning, surveying and mapping engineering, etc.

[0003] Current boundary hole repairing methods mainly fall into two categories:

[0004] Traditional manual repair: through supplementary image or manual editing of hole area point cloud, although the accuracy can be guaranteed, the field work is increased by 30%-50%, and it is highly dependent on manual experience, and the efficiency is extremely low, which cannot meet the demand of large-scale automatic modeling.

[0005] Automatic repair method:

[0006] Voxel method: point cloud is converted into signed distance field, and the hole is filled through volume diffusion, but a large amount of voxel data needs to be processed, the memory occupation is large, and it is only suitable for small-scale model.

[0007] Surface fitting method: based on the boundary information of the hole, the minimum area surface is fitted, which is suitable for closed hole, but the processing capacity for boundary hole (especially the open boundary of block edge) is weak, and the success rate of repair is less than 60%.

[0008] Aggregation repair method: such as the document disclosed in the prior art: Han, J., & Shen, S. (2019). Scalable point cloud meshing for image-based large-scale 3D modeling. Visual Computing for Industry, Biomedicine, and Art, 2(1), 1-9; the disclosed content is as follows: step 1: based on large-scale point cloud data obtained from oblique images, the point cloud data is divided into blocks using a regular grid, and an octree index is established to uniformly manage all block grids, such as Figure 1 (a) in the foregoing, Figure 1 (b) andFigure 1 Fig. 2 (c) shows;

[0009] Step 2: Take each sub-block grid as a processing unit, and based on the point cloud within the block, reconstruct the terrain surface within the block using the global optimal triangulation network construction algorithm. In this process, the boundary line does not participate in the network construction, but only serves as the triangulation network reconstruction range line;

[0010] Step 3: Complete the surface reconstruction in each sub-block grid in turn;

[0011] Step 4: Merge the triangulation surfaces of each sub-block unit based on the octree index, thereby restoring the boundary holes caused by sub-blocking to closed holes, as shown in Fig. 2 (d); Figure 1

[0012] Step 5: Use the half-edge data structure to identify whether there are holes in the sub-block grid. If there are, use the fast and robust minimum area method to reconstruct and subdivide the triangulation network within the closed hole, thereby achieving hole repair.

[0013] By aggregating the data of the sub-blocks, the boundary holes are converted into closed holes, which is a relatively mature automatic solution. However, its core relies on "global data fusion", which has exposed significant defects in practical applications.

[0014] The existing automatic methods have the following problems that cannot be ignored in large-scale oblique photogrammetry three-dimensional reconstruction:

[0015] Efficiency and memory bottleneck:

[0016] To aggregate boundary holes, multiple complete Mesh data (million to billion triangular facets) need to be loaded, resulting in a significant increase in calculation time and memory consumption. When the area is >2km 2 , the data volume exceeds the upper limit of the single machine memory, and the program directly crashes, making it impossible to complete the repair.

[0017] Limited applicability:

[0018] Insufficient type coverage: Only internal boundary holes (Class I) formed by adjacent sub-blocks can be processed, and external boundary holes (Class II) located at the edge of the model without adjacent sub-blocks cannot be repaired at all. In actual engineering, the proportion of Class II holes is as high as 15%-20% (such as urban edge water and mountain edge).

[0019] Poor form adaptability: For complex form holes such as hollow type (no adjacent surface) and island type (surrounded by isolated surface), aggregation is easy to form "pseudo-closed boundary", and the success rate of repair is <30%. Holes with an area of ≥1000m 2 have a decline in surface fitting accuracy of more than 40% due to weak boundary constraints.

[0020] Precision defects: ​

[0021] The Euclidean distance between the central area of the hole after polymerization and the boundary is large (>50m), and the elevation constraint of the boundary node to the center is weakened, which causes the curved surface to deviate from the real terrain characteristics. SUMMARY

[0022] In view of the above problems, the present application aims to provide a three-dimensional Mesh model hole repairing system and method considering the curvature characteristics of the scene structure, which aims to solve the problems of low efficiency, poor applicability and insufficient precision in boundary hole repairing in large-scale oblique photography three-dimensional reconstruction.

[0023] The three-dimensional Mesh model hole repairing method considering the curvature characteristics of the scene structure comprises the following steps:

[0024] S1: identifying the boundary hole in the three-dimensional Mesh model, and classifying the boundary hole according to the morphology and location characteristics;

[0025] S2: constructing a hierarchical structure of the boundary hole based on the topological relationship of the block grid;

[0026] S3: repairing the boundary hole layer by layer based on the hierarchical structure, including converting the boundary hole into a closed hole and fitting a curved surface;

[0027] S4: smoothing the triangular net of the adjacent boundary of the repaired area and the original model, so that the triangular net naturally transitions.

[0028] Preferably, in step S1, the identification of the boundary hole is realized by a half-edge data structure: traversing the triangular net edge of the Mesh model, if a certain edge is only associated with one triangle, it is determined as a boundary edge; tracing the boundary edge in a predetermined direction to form a closed loop, if the loop contains the block grid boundary, it is determined as a boundary hole.

[0029] Preferably, in step S1, the classification comprises:

[0030] According to the location characteristics, it is divided into internal boundary hole and external boundary hole;

[0031] According to the morphological characteristics, it is divided into at least four subcategories, including hollow type, half-enclosed type, through type and island type.

[0032] Preferably, in step S2, the construction of the hierarchical structure comprises the following steps:

[0033] S21: indexing the block grid, and clustering the boundary holes with topological association into at least one cluster;

[0034] S22: according to the spatial position of the boundary hole in the cluster, dividing it into a hierarchical structure from the outer layer to the inner layer.

[0035] Preferably, in step S21, the determination condition of the topological association is that the two boundary holes are in a shared boundary of a block grid, and the shortest Euclidean distance of the hole boundary is less than a preset threshold.

[0036] Preferably, in step S3, the process of converting the boundary hole into a closed hole includes a topological closure process, which is realized by constructing a buffer zone based on the boundary characteristics of the boundary hole, selecting a grid node on the buffer zone boundary, and forming a topologically closed polygon boundary.

[0037] Preferably, in step S3, the fitting surface includes an elevation projection conversion, specifically: projecting the boundary node to the grid boundary to generate a new node, calculating the average value of the centroid elevation of the boundary node associated triangle patch, and assigning the average value to the corresponding new node.

[0038] Preferably, in step S3, the fitting surface is fitted to the surface in the closed hole, thereby realizing hole repair, specifically including the following steps:

[0039] S31: Hole triangulation: after the original grid is topologically closed, a closed hole is generated, and the hole is triangulated using the maximum dihedral angle and minimum area criteria;

[0040] S32: Grid subdivision: the triangulated grid is subdivided by adding a barycentric point, and the specific subdivision steps are:

[0041] S321: Calculate the length of the connection edge of each vertex of the hole boundary and the average length of the connection edge of the associated first-order neighborhood vertex ;

[0042] S322: Calculate the barycenter and the length factor of the new triangular grid ;

[0043] S323: (1)

[0044] wherein, is the length of the connection edge of the vertex, is a density control factor, the greater the density factor, the denser the subdivided grid, and the default ; if the conditions of formula (1) are met, the triangular grid is subdivided, otherwise it is not subdivided; then the subdivided triangular grid is optimized using the Delaunay criterion, and if the Delaunay criterion is not met, edge exchange processing is performed; finally, the above steps are repeated until there is no new grid and grid edge exchange processing;

[0045] S33: Grid adjustment: based on the connection relationship of the newly added repair grid, a circle of inner layer ring is extracted, the outermost ring is adjusted first, then the next outer ring is adjusted, layer by layer adjustment is adopted, distance reciprocal weighted average is used for position adjustment, the first-order neighbor point and the second-order neighbor point of the newly added node in the outermost layer are calculated; the weight of each adjacent point is calculated; the weighted average algorithm is used to adjust the position from outside to inside layer by layer, the adjustment function is as follows:

[0046] (2)

[0047] In the formula: , is the adjacent node of the newly added node ;

[0048] S33: Grid smoothing: Laplace smoothing is adopted, and the following formula is adopted:

[0049] (3)

[0050] In the formula, is the cotangent weight, are the angle values of the two adjacent triangles corresponding to the edge , indicates the deviation of the vertex from the adjacent triangular mesh, which is called Laplace harmonic umbrella operator, and the second-order umbrella operator can be obtained by recursion;

[0051] (4)

[0052] In the formula, indicates that the deviation of the vertex from the triangular mesh is equal to the average deviation of the adjacent triangular mesh, which is equivalent to reducing the optimization of the grid to a linear equation, and finally completes the smoothing processing.

[0053] Preferably, in step S4, the smoothing processing of the triangular mesh of the adjacent boundary of the repair area and the original model includes the following: the Laplace smoothing method is adopted to smooth the shared boundary elevation, and the elevation of each node is assigned as the average value of the elevations of the nodes within a certain buffer range.

[0054] For each repaired point, let its original elevation be , and define its updated elevation value as :

[0055] (5)

[0056] In the formula, is the smoothing sparsity, , is defined as follows:

[0057] ​(6)

[0058] wherein, is the k-th neighbor point of the point, is the number of neighbor points.

[0059] The three-dimensional Mesh model hole repairing system considering the curvature characteristics of scene structure, comprises:

[0060] A hole identification and classification module is configured to perform the step S1.

[0061] A hierarchical structure construction module is configured to perform the step S2.

[0062] A layer-by-layer repairing module is configured to perform the step S3, which comprises a topological closure unit, an elevation projection unit and a curved surface fitting unit.

[0063] A boundary smoothing module is configured to perform the step S4, which uses the formulas (5) and (6) to realize the smoothing processing.

[0064] Compared with the prior art, the three-dimensional Mesh model hole repairing system and method considering the curvature characteristics of scene structure have the following beneficial effects:

[0065] 1. Overcoming the bottleneck of large-scale repairing efficiency

[0066] The block grid independent processing strategy is adopted to avoid global data aggregation, and the hierarchical layer-by-layer repairing logic is combined to significantly reduce the computational complexity and memory consumption, so that the boundary hole repairing of an ultra-large scale region can be stably supported, and the size limitation of the traditional method is broken through.

[0067] 2. Comprehensive coverage of complex hole types

[0068] Through the fine classification of location and shape (internal / external boundary holes, hollow type / semi-enclosed type / piercing type / island type four subcategories), the hierarchical clustering is combined to construct the repairing relationship of outer constraint and inner layer, so that the complex shape holes and external boundary holes can be effectively repaired, and the applicability of the method is greatly expanded.

[0069] 3. Improving the accuracy and naturalness of curved surface fitting

[0070] The innovative topological closure and elevation projection collaborative optimization mechanism is used to give reasonable topological and elevation constraints to the hole boundary; the curvature-aware Laplace smoothing algorithm is adopted to keep the curvature consistency of the repaired area and the original model, to ensure the natural transition of the curved surface, and to meet the high-precision modeling demand.

[0071] ​(1) In terms of repair success rate: for the larger proportion of closed holes, the repair results of the method of the application and the existing method are consistent; for class I boundary holes, the repair success rate of the method of the application is improved by 12.4% compared with the mature existing method, in addition, the method of the application makes up for the deficiency of the existing method that cannot complete the repair of class II boundary holes, and has better applicability.

[0072] (2) In terms of repair efficiency: the existing method can only complete hole repair in a small range (≤2km 2 ), when the experimental area is large (>2km 2 ), the method will cause computer downtime due to too large amount of data to be processed, and cannot complete hole repair, in the repairable experimental range, the time consumption of the existing method is about 4-5 times that of the method of the application. In addition, in terms of memory consumption, in the repairable experimental range, the memory consumption of the existing method is about 7-21 times that of the method of the application.

[0073] (3) In terms of repair accuracy: compared with the existing method, the method of the application improves the average height difference by 37.5% and the standard deviation of the height difference by 38.9%, and the overall height accuracy is high and the error is small.

[0074] 4, Promote the landing of engineering application

[0075] Through efficiency improvement and applicability expansion, reduce manual intervention and field cost, shorten the modeling cycle, and can be widely applied to three-dimensional reconstruction of complex terrain and large-scale area, promote the industrialized application of oblique photography technology in the field of digital twin and intelligent surveying and mapping. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 It is a boundary hole repair principle diagram of the existing method, in which (a) represents an original closed hole, (b) represents a block grid, (c) represents an unclosed hole generated by block division, and (d) represents a closed hole generated by aggregation of the unclosed hole;

[0077] Figure 2 It is a hole schematic diagram, in which (a) represents a half-edge data structure, and (b) represents an unclosed hole;

[0078] Figure 3 It is a rough classification schematic diagram of unclosed holes in the application, in which (a) represents an internal unclosed hole, and (b) represents an external unclosed hole;

[0079] Figure 4 It is a fine classification schematic diagram of unclosed holes in the application, in which (a) represents a hollow type, (b) represents a half-enclosed type, (c) represents a through type, and (d) represents an island type;

[0080] Figure 5For the hierarchical relationship diagram in the present application, in the diagram, (a) represents an unclosed hole, and (b) represents a split grid;

[0081] Figure 6 For the topologically closed diagram in the present application, in the diagram, (a) represents type 1, (b) represents type 2, and (c) represents type 3;

[0082] Figure 7 For the topologically closed diagram of the unclosed hole in the present application, in the diagram, (a) represents a hollow type, (b) represents a half-enclosed type, (c) represents a through type, and (d) represents an island type;

[0083] Figure 8 For the elevation propagation diagram in the present application, in the diagram, (a) represents identifying each boundary node, and (b) represents elevation transfer assignment;

[0084] Figure 9 For the hole repairing diagram in the present application, in the diagram, (a) is an original grid, (b) is a topologically closed hole, (c) is a hole triangulation, (d) is a grid subdivision, (e) is an adjacent point extraction, and (f) is a height adjustment;

[0085] Figure 10 For the angle diagram of the cotangent weight in the present application;

[0086] Figure 11 For the common boundary and elevation difference in adjacent holes in the present application;

[0087] Figure 12 For the hole repairing result diagrams of the two methods in the present application; in the diagram, (a) represents the Han method, and (b) represents the method of the present application;

[0088] Figure 13 For the hole distribution after repairing in the present application; in the diagram, (a) represents hole distribution, and (b) represents hole repairing;

[0089] Figure 14 For the time consumption statistical histogram of repairing in the present application;

[0090] Figure 15 For the memory consumption statistical histogram in the present application;

[0091] Figure 16 For the experimental data result diagram in the present application; in the diagram, (a) represents an original grid, (b) represents artificial hole digging, and (c) represents hole segmentation;

[0092] Figure 17 For the experimental data result diagram of the Han method in the prior art; in the diagram, (a) represents hole aggregation, (b) represents hole repairing, and (c) represents a hole local detail diagram;

[0093] Figure 18 The figure shows the experimental data results of the method of the present invention; in the figure, (a) represents hole aggregation, (b) represents hole repair, and (c) represents a local detail of the hole.

[0094] Figure 19 This is a comparison chart of repair accuracy. Detailed Implementation

[0095] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0096] Example 1:

[0097] This embodiment discloses a method for repairing holes in a 3D mesh model that takes into account the curvature characteristics of the scene structure, including the following steps:

[0098] S1: Identify boundary holes in the 3D Mesh model and classify the boundary holes according to their morphological and locational characteristics;

[0099] Specifically, this embodiment uses a half-edge data structure for boundary hole identification. The half-edge data structure is used to organize the triangular faces within the block to quickly identify adjacent triangles associated with a particular edge, such as... Figure 2 As shown in (a), edge e 0,1 There are two halves of h facing opposite directions. 0,1 and h 1,0 This indicates that it is associated with two triangles; if a side is associated with only one triangle, it is defined as a boundary edge. Boundary edge tracing is performed along a fixed direction (clockwise or counterclockwise). If the resulting loop contains a segmented grid boundary, then the hole is a boundary hole. Figure 2 As shown in (b).

[0100] Based on the location of boundary holes in the reconstruction area, boundary holes are first divided into two categories: Category I—internal boundary holes, that is, the boundary of the grid blocks contained in the boundary hole is not the boundary of the reconstruction area, such as... Figure 3 As shown in (a); Type II—external boundary holes, that is, the boundary holes contain grid blocks whose boundaries are the boundaries of the reconstructed region, such as... Figure 3 As shown in (b).

[0101] Furthermore, based on the morphological characteristics of the boundary holes, they are divided into four subcategories:

[0102] Type A: Hollow type: Boundary holes fill the entire block grid area, meaning there are no adjacent reconstructed surfaces such as... Figure 4 As shown in (a);

[0103] Class B: semi-enclosed type: the boundary hole is located at one side of the sub-block grid area, and the other side is a reconstructed surface, as shown in Fig. 2 (b); Figure 4

[0104] Class C: through type: the boundary hole is located in the middle of the sub-block grid area, and both sides are reconstructed surfaces, as shown in Fig. 2 (c); Figure 4

[0105] Class D: island type: the boundary hole is located in the middle of the sub-block grid area, and both sides are reconstructed surfaces, and one of them (or both) is island-shaped, as shown in Fig. 2 (d). Figure 4

[0106] S2: constructing a hierarchical structure of the boundary hole based on the topological relationship of the sub-block grid;

[0107] Specifically, first, the grid index is established to organize the original sub-block grid. For two boundary holes, if the sub-block units where they are located are adjacent to each other and there is a shared grid boundary between the holes, they are clustered and identified, but no merging operation is performed. The above process is iterated until all boundary holes participate in the calculation, and the cluster of each boundary hole is obtained. Second, the boundary holes in each cluster are divided into levels, as follows: (1) the boundary holes located at the outermost side of the cluster are the first level, which can be determined by identifying the boundary edges in the initial triangular mesh, i.e. the boundary holes where the boundary edges are located are in the first level; (2) the holes that share the boundary with the first level boundary holes and do not belong to the first level are the second level holes; (3) in this way, the nth level holes are obtained, thereby establishing the hierarchical relationship between the boundary holes. As shown in Fig. 2, for the boundary hole cluster in Fig. 2 (a), the hierarchical relationship therebetween is shown in Fig. 2 (b). Figure 5 Figure 5 Figure 5

[0108] S3: repairing the boundary holes layer by layer based on the hierarchical structure, including converting the boundary holes to closed holes and fitting the surface;

[0109] Specifically, after establishing the hierarchical relationship of the boundary holes, the outermost holes are started (the first level), and the holes are repaired layer by layer from outside to inside. There are two key problems in the repair process: (1) how to convert the boundary holes to closed holes, i.e. how to realize the topological closure of the boundary; (2) how to reasonably fit the surface of the holes in the block, i.e. how to assign appropriate elevation values to the grid nodes near the grid boundary. Therefore, a unified topological closure principle is established for the closure processing of multiple types of boundary holes, and the elevation value projection conversion is performed to complete the layer-by-layer repair of the boundary holes.

[0110] ​​​​​​Firstly, the boundary of any hole is divided into two parts: curved surface boundary and grid boundary, the former is the boundary edge of the original triangular curved surface constituting the hole, and the latter is the boundary of the block grid constituting the hole. Although the grid boundary is part of the boundary hole, the connection between it and the curved surface boundary part is not topologically closed, so it is difficult to convert it into a closed hole for repair. The present application takes into account the topological constraints to close the boundary, analyzes the boundary characteristics of four types of boundary holes, and their topological closure can be summarized as follows: (1) the first and last endpoints of the curved surface boundary edge are located on both sides of the grid boundary, then take the grid boundary as the center, and the distances from the first and last endpoints to the grid boundary as the constraints, respectively, to establish a buffer zone on both sides of the grid boundary, and then select the grid nodes on the boundary of the buffer zone to form a topologically closed triangular grid, as shown in (a) of Figure 6 ; (2) one endpoint of the curved surface boundary edge is located on the grid boundary, and the two boundary edge nodes associated with the endpoint are located on both sides of the grid boundary, then take the grid boundary as the center, and the distances from the two boundary edge nodes associated with the endpoint to the grid boundary as the constraints, respectively, to establish a buffer zone on both sides of the grid boundary, and then select the grid nodes on the boundary of the buffer zone to form a topologically closed triangular grid, as shown in (b) of Figure 6 ; (3) one endpoint of the curved surface boundary edge is located on the grid boundary, and the two boundary edge nodes associated with the endpoint are located inside the grid boundary, then take the grid boundary as the center, and the average value of the two boundary edge nodes associated with the endpoint as the distance constraint, to establish a buffer zone outside the grid boundary, and then select the grid nodes on the boundary of the buffer zone to form a topologically closed triangular grid, as shown in (c) of Figure 6 .

[0111] Based on the above topological closure rules, different closure strategies are used to topologically close different types of unclosed holes. For A type-“hollow type” unclosed hole, a buffer zone is constructed based on the segmentation nodes inside and outside the grid boundary, and then topologically closed, as shown in (a) of Figure 7 . For B type-“half-enclosed type” unclosed hole, a buffer zone is constructed based on the hole boundary and the segmentation nodes inside and outside the grid boundary, and then topologically closed, as shown in (b) of Figure 7 . For C type-“penetrating type” unclosed hole, a buffer zone is constructed based on the hole boundary inside and outside the grid boundary, and then topologically closed, as shown in (c) of Figure 7 . For D type-“island type” unclosed hole, a buffer zone is constructed inside and outside the shortest place between the island boundary and the hole boundary, and then topologically closed, as shown in (d) of Figure 7 .

[0112] To ensure the accuracy of the fitted surface for the holes within the block, appropriate elevation values ​​need to be assigned to the grid nodes near the grid boundary. Therefore, an elevation value projection transformation is performed. First, the boundary nodes are projected onto the grid boundary, and the projected points are used as new nodes on the grid boundary to ensure that the new triangular faces have similar compactness to the original triangular faces. Figure 8 As shown in (a); next, identify the boundary triangular facets associated with each boundary node, and calculate the average elevation of the centroid of each boundary triangular facet; finally, assign the elevation value associated with each boundary node to its corresponding newly added node, as shown in (a). Figure 8 As shown in (b). To improve the accuracy of elevation assignment, this invention first uses the four corner points of the block grid as dividing points to divide the block grid boundary into four parts, and then uses the above algorithm to obtain the new nodes of each part boundary and their corresponding elevation values.

[0113] In the process of repairing boundary holes, the elevation of the grid boundary in each block unit is first assigned in the first layer, and then this value is passed to the second layer. The propagation process is consistent with the intra-block elevation propagation process. Based on the above algorithm, the boundary hole can be converted into a topologically closed hole, and the grid boundary has suitable nodes for triangulation. Therefore, the surface inside the closed hole can be fitted according to the minimum and maximum dihedral angle and area method, thereby realizing the hole repair.

[0114] The specific steps are as follows:

[0115] (1) Steps for triangulation of holes: as follows Figure 9 As shown in (a), the original mesh generates closed holes after topological closure, as shown in (a). Figure 9 As shown in (b), the hole is triangulated using the "maximum dihedral angle and minimum area" criterion. The specific triangulation rules are: (1) Prioritize adding triangular faces that form the maximum dihedral angle with the neighboring triangulation network; (2) Then consider adding triangular faces with the minimum area. Iterate through the holes to triangulate all vertices, as shown in (b). Figure 9 As shown in (c).

[0116] (2) Mesh subdivision steps: Triangulation meshes are subdivided by adding centroids, such as... Figure 9 As shown in (d), the specific subdivision steps of the grid are as follows: First, calculate each vertex of the hole boundary. Average length of connecting edges of associated first-order neighborhood vertices Then calculate the newly added triangular mesh. center of gravity and length factor If the conditions of formula (1) are met simultaneously, then for the triangular mesh... Subdivision, otherwise not; then the Delaunay criterion is used to optimize the subdivision of the triangular mesh, if not meet the Delaunay criterion, then the edge exchange processing; finally, the above steps are repeated until there is no new grid and grid edge exchange processing.

[0117] (1)

[0118] In the formula, is the length of the edge connected to the vertex, is the density control factor, the greater the density factor, the more intensive the subdivision grid, the present application defaults .

[0119] (3) Grid adjustment step: the newly added repair grid adopts the adjustment sequence from outside to inside, and extracts a circle of inner layer based on the connection relationship of the newly added repair grid. The outermost ring is adjusted first, and then the next outer ring is adjusted. The position is adjusted by using distance reciprocal weighted average. The farther the distance from the newly added node, the smaller the influence on the position of the newly added node. First, the first-order neighbor points (blue nodes) and the second-order neighbor points (green nodes) of the outermost newly added nodes are calculated, as shown in (e) of Figure 9 ; secondly, the weight of each adjacent point is calculated; finally, the position is adjusted layer by layer from outside to inside by using the weighted average algorithm, the dark gray nodes are adjusted first, and then the light gray nodes are adjusted, as shown in (f) of Figure 9 . The adjustment function is as follows.

[0120] (2)

[0121] In the formula , is the adjacent node of the newly added node , such as the node pointed by the yellow arrow in (f) of Figure 9 .

[0122] (4) Grid smoothing step: in order to make the newly added grid more smooth, the grid needs to be fairing. The global smoothing based on curvature can better make the newly added grid keep the original shape of the model. The relatively popular mesh fairing method at present is to take the weighted barycenter of the one-ring neighborhood of each point as the position of the new vertex, which is defined as the weighted umbrella operator. In fact, it is to solve the Laplace matrix.

[0123]

[0124] In the formula, is the first-order neighborhood vertex of the vertex ; is the weight value of the edge.

[0125] The difference between the curvature-based smoothing and the above-mentioned Laplace smoothing is that the curvature-based smoothing is along the vertex Use the normal vector to move the vertex The conventional Laplace smoothing, on the other hand, is to smooth the vertex... Move to a position similar to the center of gravity. It can be deduced that by selecting appropriate Laplace weights, the calculation formula is shown in (4), and Laplace smoothing can achieve the same effect as curvature-based smoothing.

[0126] (3)

[0127] In the formula, For cotangent weight, They are the edges The angle values ​​of the two corresponding adjacent triangles, such as Figure 10 As shown. Represents vertices The deviation from the adjacent triangular mesh is called the Laplace harmonic umbrella operator. The second-order umbrella operator can be obtained through recursion.

[0128] (4)

[0129] In the formula, This means that the deviation between a vertex and a triangular mesh is equal to the average deviation of adjacent triangular meshes, which is equivalent to reducing mesh optimization to linear equations and completing the final smoothing process.

[0130] S4: Smooth the triangular mesh at the boundary between the repaired area and the original model to make the triangular mesh transition naturally.

[0131] Specifically, after layer-by-layer processing, the repair of all boundary holes within the segmented units is completed. However, because the processing is done layer by layer, such as Figure 11 As shown, there will be an elevation difference at the common boundary of two adjacent holes in the same layer. Therefore, it is necessary to smooth the common boundary of adjacent holes in the same layer.

[0132] Laplacian smoothing is widely used for surface smoothing. It can homogenize the mesh while preserving the local geometric details of the surface. It is also simple to compute and has a short computation time. Therefore, this invention uses the Laplacian smoothing method to smooth the shared boundary elevation. The principle is to assign the elevation of each node to the average value of the node elevations within a certain buffer range.

[0133] For each point to be repaired, let its original elevation be... Define its updated elevation value as :

[0134] (5)

[0135] In the formula, To achieve smooth sparseness, , The definition is as follows:

[0136] (6)

[0137] In the formula, is the kth neighborhood point of the point, is the number of neighborhood points.

[0138] Experiment and analysis

[0139] The experimental area is part of the built-up area of a certain city, the measurement area is 5.2km x 7.8km, 29 flight strips of UltraCamOspreyPrima (UCOp) cameras with 5 lenses (1 orthoview + 4 side view) are used to obtain 11795 photos, and the total data amount is 2.08TB. The reconstruction range is about 28km 2 The reconstruction area is divided into 449 sub-areas with a grid size of 250m x 250m, and the running environment is a standard personal computer with Windows 10 64-bit operating system, Intel Xeon(R) E3-1535M CPU with a main frequency of 3.10GHz, and 64G memory. The method of the present application is compared and analyzed with the existing Han et al. (2019) method (hereinafter referred to as Han method) to verify the effectiveness and superiority of the method of the present application. The experiment is divided into three parts: comparison and verification of repair success rate, comparison and verification of repair efficiency, and comparison and verification of repair results.

[0140] (1) Comparison of repair success rate in local range of experimental area

[0141] The middle position (more water) of 2km 2 area in the reconstruction area is selected for comparison and experiment of repair success rate of non-closed hole, the repair results of the two methods are shown in Figure 12 (a) and Figure 12 (b), the area range of non-closed hole in the experimental area is [1242.99m 2 , 13703.97m 2 ], and the distribution and repair of closed hole, type I and type II non-closed hole are shown in Table 1 and Table 2.

[0142] Table 1 Distribution and repair of closed hole and type I non-closed hole

[0143]

[0144] Table 2 Distribution and repair of type II non-closed hole

[0145]

[0146] ​It can be found from Table 1 that there are 16 closed holes in the experimental area. Since the minimum area method is used in the hole repairing process of the method of the present application, both the method of the present application and the Han method can repair the closed holes, and the repairing rate is 100%. For the non-closed holes of Type-A, the number of Type-B is the largest, which is 17, and the number of Type-C is the smallest, which is 0. In general, the repairing success rate of the Han method is 50%, and the repairing success rate of the method of the present application is 100%, and the repairing success rate of the method of the present application is higher. The difference between the two methods mainly lies in the processing of Type-A and Type-B holes. According to the actual data, it is found that the 2 Type-A non-closed holes and 8 Type-B non-closed holes which are repaired unsuccessfully by the Han method form a large non-closed hole, which leads to the failure of repairing.

[0147] It can be seen from Table 2 that, compared with the non-closed holes of Type-I, the number of non-closed holes of Type-II is obviously less, which is only 2. For the non-closed holes of Type-II, since they cannot form a closed hole with adjacent holes or there is no adjacent hole around them, the Han method cannot complete the repairing. Unlike this, the repairing success rate of the method of the present application is 100%, which shows that the method of the present application has better applicability.

[0148] (2) Comparison of repairing success rate in local range of experimental area

[0149] In the global range of 28km 2 of experimental area, the non-closed hole repairing success rate comparison experiment is carried out, and the holes and repairing results are shown in (a) and (b) of FIG. 2; the distribution and repairing of closed holes, non-closed holes of Type-I and non-closed holes of Type-II are shown in Table 3 and Table 4. Figure 13 Figure 13

[0150]

[0151]

[0152] Table 4 Distribution and repairing of non-closed holes of Type-II

[0153]

[0154] ​​​It can be found from Table 3 and Table 4 that there are 56 closed holes, 54 I-type unclosed holes and 6 II-type unclosed holes in the experimental area. Since the experimental range is too large, the Han method needs to load all the grid data with holes for hole aggregation, and the computer memory overflows, resulting in program abnormal exit, so that the hole repair cannot be completed. The method of the application does not need to perform hole aggregation, and loads the grid data with holes block by block and performs topological closure, so that the repair of all holes can be completed.

[0155] Repair efficiency comparison and verification

[0156] Six groups of areas of 0.5km 2 , 1km 2 , 1.5km 2 , 2km 2 , 2.5km 2 and 28km 2 are selected in the reconstructed area for boundary hole repair experiment, and the time consumption and memory consumption of the Han method and the method of the application are counted and compared. The counting time starts from the time when the block triangular net reconstruction is completed, and ends when the boundary hole repair is completed, mainly covering the processes of boundary hole identification, repair and smoothing.

[0157] (1) Repair time consumption comparison

[0158] The repair time consumption of the two methods in different sizes of experimental areas is shown in Table 5, and the corresponding histogram is shown in Figure 14 .

[0159] It can be found from Table 5 and Figure 14 that: (1) with the increase of the experimental area, the time consumption of the two methods shows a growth trend, but the time consumption of the method of the application is lower than that of the Han method as a whole; (2) the Han method can only complete the hole repair of a small range area (≤2km 2 ), when the experimental area is large (>2km 2 ), the method will cause computer downtime due to too large data volume to be processed, and cannot complete the hole repair; (3) within the repairable experimental range, the time consumption of the Han method is about 4-5 times that of the method of the application.

[0160] Table 5 Comparison of unclosed hole repair time consumption of two methods in different experimental areas

[0161]

[0162] Note: The counting time is in minutes.

[0163] Table 6 Comparison of unclosed hole repair memory of two methods in different experimental areas

[0164]

[0165] Note: The statistical unit is GB, and the statistical value of the method is the maximum memory consumption during the hole repairing process.

[0166] (2) Memory consumption comparison

[0167] The memory consumption statistics of the two methods in different sizes of experimental areas are shown in Table 6, and the corresponding histograms are shown in Figure 15 .

[0168] From Table 6, Figure 16 (a), Figure 16 (b) and Figure 16 (c), it can be found that: (1) As the experimental area increases, the memory consumption of both methods shows a growing trend, but the memory consumption of Han method increases significantly, and the memory consumption of the method increases slowly; (2) In a single machine environment, when the experimental area is large (>2km 2 ), Han method cannot complete hole repair due to computer downtime; (3) Within the repairable experimental range, the memory consumption of Han method is about 7-21 times that of the method.

[0169] Repair result comparison and verification

[0170] To verify the repair results and accuracy of the method, a closed hole and an unclosed hole with an area of 76112.5m 2 are artificially dug out in the area of the experimental area where the triangular mesh surface is complete, and the area is divided into four sub-areas. The closed hole is divided into four internal unclosed holes, and the unclosed hole is divided into two external unclosed holes, as shown in Figure 17 (a), Figure 17 (b) and Figure 17 (c).

[0171] Using the above hole data as input data, Han method and the method are used for hole repair, and the repair results are compared and analyzed.

[0172] (1) Repair result comparison

[0173] From Figure 17 (a), Figure 17 (b) and Figure 17 (c), it can be seen that after hole aggregation, Han method, the internal unclosed hole is converted into an internal closed hole, which can be repaired. The external unclosed hole is still an unclosed hole, which cannot be repaired. From Figure 18 (a), Figure 18 (b) and Figure 18As can be seen in (c), after the hole is closed using the method of the present invention, both the internal and external unclosed holes are transformed into closed holes, enabling hole repair. Furthermore, regarding the repair results at the boundary of the closed hole, the hole boundary line is clearly visible in the Han method, while the method of the present invention allows for a natural transition.

[0174] (2) Comparison of repair accuracy

[0175] The repair accuracy was evaluated by comparing the elevation values ​​of the repaired triangulated network nodes with those of the original nodes. In this experiment, the elevation range of the entire hole was 3.96m to 5.98m, with a difference of 2.02m. Thirty-four evenly distributed nodes were selected within the closed hole area, and the differences between the fitted elevation values ​​of the two methods and the original elevation values ​​at each node were statistically analyzed. The results are shown below. Figure 19 As shown.

[0176] Depend on Figure 19 It can be seen that the height difference of the mesh repaired by the method of this invention is smaller than that of the mesh repaired by the Han method. Compared with the original mesh, the average difference in elevation direction between the mesh repaired by the Han method and the original mesh is 16cm, and the maximum difference is 36cm; correspondingly, the average difference in elevation direction between the mesh repaired by the method of this invention and the original mesh is 10cm, and the maximum difference is 23cm. Further analysis of the standard deviation of the elevation difference shows that the standard deviation of the elevation difference for the Han method is 18cm, while the standard deviation for the elevation difference for the method of this invention is 11cm, indicating that the overall elevation accuracy of the nodes obtained by the method of this invention is higher and the error is smaller. Comparing with actual data, it was found that the closer the node is to the center of the hole, the greater its elevation difference with the original node. This is directly related to the process of hole triangulation and filling, and also indicates that the repair accuracy decreases as the size of the hole increases.

[0177] Example 2:

[0178] The 3D Mesh model hole repair system that takes into account the curvature characteristics of scene structure as described in this invention includes:

[0179] Hole recognition and classification module: Identifies boundary holes in the 3D Mesh model and classifies the boundary holes according to their shape and location characteristics;

[0180] Hierarchical structure construction module: Constructs the hierarchical structure of the boundary holes based on the topological relationship of the segmented grid;

[0181] Layer-by-layer repair module: Based on the hierarchical structure, repairs boundary holes layer by layer, including converting boundary holes into closed holes and fitting curved surfaces, including topological closure units, elevation projection units and curved surface fitting units;

[0182] The boundary smoothing module: the triangular net of the abutted boundary of the repaired area and the original model is smoothed, the triangular net is naturally transitioned, and the smoothing is realized by using formulas (5) and (6).

[0183] The system of the embodiment is based on the method of the embodiment 1, and realizes: (1) boundary hole identification and classification: the boundary hole identification is performed based on the half-edge data structure, and according to the shape and location characteristics, the boundary hole is divided into two categories and four sub-categories; (2) boundary hole hierarchical relationship calculation: the block cell net index is established, the boundary hole hierarchical relationship is calculated based on the abutted relationship between the block cells; (3) boundary hole layer-by-layer propagation repair: the boundary hole is closed based on the topological closure principle, the elevation value projection conversion is performed, and the boundary hole layer-by-layer repair is completed; (4) abutted hole smoothing processing: the abutted boundary smoothing processing is performed based on the Laplace algorithm, so that the natural transition of the triangular net between the block cells is realized.

[0184] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can make equivalent replacement or change according to the technical scheme and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A hole repairing method of a three-dimensional mesh model considering a curvature feature of a structure of a scene, characterized in that, The method comprises the following steps: S1: identifying boundary holes in a three-dimensional mesh model, and classifying the boundary holes according to morphological and location characteristics; S2: constructing a hierarchical structure of the boundary holes based on the topological relationship of the block grid; S3: repairing the boundary holes layer by layer based on the hierarchical structure, including converting the boundary holes into closed holes and fitting a curved surface; S4: smoothing the triangular mesh of the original model adjacent to the boundary of the repaired area, so that the triangular mesh transitions naturally. In step S2, the construction of the hierarchical structure comprises the following steps: S21: indexing the block grid, and clustering boundary holes with topological association into at least one cluster; S22: dividing the boundary holes in the cluster into a hierarchical structure from the outer layer to the inner layer according to the spatial position of the boundary holes. In step S3, the fitting of the curved surface is to fit the curved surface in the closed hole to achieve hole repair, which specifically comprises the following steps: S31: hole triangulation: after the original mesh is topologically closed to generate a closed hole, the hole is triangulated according to the maximum dihedral angle and minimum area criteria; S32: mesh subdivision: the triangulated mesh is subdivided by adding a barycentric point, and the specific subdivision steps are as follows: S321: Calculate each vertex of the hole boundary Average length of connecting edges of associated first-order neighborhood vertices ; S322: Calculate the new triangular mesh gravity center and length factor ; S323： （1） in, Center of gravity To the top The vector value, The length of the edge connecting the vertex. This is the density control factor. The larger the density factor, the denser the subdivision mesh. (Default) If the conditions of formula (1) are met simultaneously, then for the triangular mesh... If the subdivision is not performed, then the subdivided triangular mesh is optimized using the Delaunay criterion. If the Delaunay criterion is not met, then edge swapping is performed. Finally, the above steps are repeated until no new meshes are added and no mesh edge swapping is performed. S33: grid adjustment: based on the connection relationship of the newly added repair mesh, a circle of inner rings is extracted, the outermost ring is adjusted first, then the next outer ring is adjusted, and layer-by-layer adjustment is performed, and the position is adjusted by using distance reciprocal weighted average, the first-order neighbor points and the second-order neighbor points of the newly added nodes in the outermost layer are calculated; the weights of each adjacent point are calculated; the position is adjusted layer by layer from the outside to the inside by using the weighted average algorithm, and the adjustment function is as follows: （2） In the formula: is a newly added node, , is a newly added node adjacent node of the newly added node; S34: grid smoothing: Laplace smoothing is adopted, and the following formula is used: （3） In the formula, is cotangent weight, respectively, the edges corresponding to the angle value of two adjacent triangles, denotes the vertex Deviation from the adjacent triangular mesh, called Laplace harmonic umbrella operator, through recursion, the second order umbrella operator can be obtained; （4） wherein is cotangent weight, denotes that the deviation of the vertex from the triangular mesh is equal to the average deviation of the adjacent triangular mesh, which is equivalent to reducing the optimization of the mesh to a linear equation to complete the final smoothing.

2. The method of claim 1, wherein, In step S1, the identification of the boundary hole is realized by using half-edge data structure: traversing the triangular mesh edge of the mesh model, if a certain edge is associated with only one triangle, it is determined as a boundary edge; a closed loop is formed by tracking the boundary edge in a predetermined direction, if the loop contains the block grid boundary, it is determined as a boundary hole.

3. The method of claim 1, wherein, In step S1, the classification comprises: According to the location characteristics, it is divided into internal boundary holes and external boundary holes; According to the morphological characteristics, it is divided into at least four subcategories, including hollow type, half-enclosed type, through type and island type.

4. The method of claim 1, wherein, In step S21, the determination condition of the topological association is that the block grid where the two boundary holes are located has a shared boundary, and the shortest Euclidean distance of the hole boundary is less than a preset threshold.

5. The method of claim 1, wherein, In step S3, the process of converting the boundary hole into a closed hole comprises topological closure processing, which is realized by the following way: a buffer zone is constructed based on the boundary characteristics of the boundary hole, and a network node is selected on the boundary of the buffer zone to form a topologically closed polygon boundary.

6. The method of claim 5, wherein the method further comprises: In step S3, the fitting of the curved surface comprises elevation projection conversion, specifically: projecting the boundary nodes to the grid boundary to generate new nodes, calculating the average value of the centroid elevations of the triangular facets associated with the boundary nodes, and assigning the average value of the centroid elevations to the corresponding new nodes.

7. The method of claim 1, wherein, The step S4 specifically comprises the following: using a Laplace smoothing method to smooth the shared boundary elevations, and assigning the elevation of each node as the average of the elevations of the nodes within a certain buffer range of the node. For each patched point, let its original elevation be , and define its updated elevation value as : （5） wherein is defined as follows: , is defined as follows: （6） In the formula, is the number of neighboring points of the point, is the number of neighboring points of the point, is the number of neighboring points of the point.

8. A system for hole repairing of a three-dimensional mesh model considering curvature features of scene structure, based on the hole repairing method of a three-dimensional mesh model considering curvature features of scene structure according to any one of claims 1-7, characterized in that, The method comprises the following steps: a hole identification and classification module for performing the step S1; a hierarchy construction module for performing the step S2; a layer-by-layer repairing module for performing the step S3, including a topological closure unit, an elevation projection unit and a curved surface fitting unit; a boundary smoothing module for performing the step S4 to realize the smoothing of the triangular mesh.

Images