A method for repairing a three-dimensional model of a city wall site

Abstract

The application discloses a kind of city wall ruins three-dimensional model repair methods.It includes the following steps: S1: separating from original model to be repaired area;S2: the mesh subdivision of above-mentioned to be repaired area model is carried out;S3: in above-mentioned to be repaired area model, interactive intercepts city brick dense model;Interactive intercepts the center point cloud of to be repaired local area and saves;S4: based on the dense point cloud segmentation of above-mentioned center point cloud, calculate city brick basic position line;S5: if the position line of the edge portion of to be repaired local area and city brick basic position line are not on the same straight line, modify corresponding city brick basic position line;S6: generate PLY and OBJ two kinds of format three-dimensional mesh model.Advantages are: the geometric fineness of the city wall ruins class three-dimensional repair model generated is higher, texture is clearer, real sense is higher, can more clearly, completely and intuitively present the structure of bag brick wall, provide important basis for subsequent city wall ruins protection and show, simultaneously have universality.

Description

Technical Field

[0001] This invention relates to a method for restoring a three-dimensional model of a city wall ruin, belonging to the field of computer graphics technology. Background Technology

[0002] Oblique photogrammetry is used to acquire 3D mesh models suitable for cultural relic preservation scenarios. OBJ and OSGB are two common 3D model / scene file formats. OBJ is a text format primarily used to store the geometric data (vertices, faces, texture coordinates, etc.) and material information of a single 3D model; it is one of the most widely used 3D model formats and is compatible with most 3D software. OSGB is a binary scene format used to store complex 3D scenes (including models, lights, cameras, animations, hierarchical structures, etc.), optimizing data storage efficiency and loading speed. During data processing, the restoration software uses the PLY format for the mesh model of the area to be restored. Elements include: vertices, which store 3D coordinates and color attributes; and triangular meshes (faces), which store the vertex indices of the triangular meshes.

[0003] However, model restoration often suffers from incompleteness and lack of detail due to issues such as missing data, noise, and geometric deviations. This is especially true for reconstructions of city wall models in low-lying areas or those soaked in groundwater, where the results are often poor. For example, the 3D model of the Iron Wall City West Gate, generated by the Smart3D software, is in OSGB format, which is unsuitable for the restoration software described in this paper. OSGBLAB software can be used to convert it to OBJ format. Furthermore, current modeling software such as SVSModeler, DP-Modeler, and 3ds Max typically use triangular meshes to model the entire building walls and perform texture mapping (i.e., using large-area triangular meshes for geometric modeling and texture mapping). The low geometric precision of these 3D mesh models results in a lack of realism in the restored 3D model, failing to clearly, completely, and intuitively represent the structure of the brick-clad walls. Therefore, achieving the goal of "restoring the old as it was" has become a pressing technical challenge. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method for restoring a 3D model of a city wall ruin that has a high degree of realism in the repaired part of the 3D model, can more clearly, completely and intuitively show the structure of the brick wall, and achieve the purpose of "restoring the old as before".

[0005] To address the aforementioned problems, the present invention provides a method for restoring a three-dimensional model of a city wall ruin, comprising the following steps:

[0006] S1: The original model of the city wall ruins is in OBJ format. The area to be restored is separated from the original model.

[0007] S2: Subdivide the mesh of the above-mentioned area to be repaired;

[0008] S3: In the above model of the area to be repaired, interactively extract the dense model of the city bricks, add it to the city brick library and save it; interactively extract the center point cloud of the local area to be repaired and save it.

[0009] S4: Based on the dense point cloud segmentation of the central point cloud of the local area to be repaired, calculate the basic position line of the city bricks.

[0010] S5: If the position line of the edge part of the local area to be repaired is not on the same straight line as the basic position line of the city brick, then interactively add, delete or modify the position line of the city brick edge, and modify the corresponding basic position line of the city brick.

[0011] S6: Based on the brick location lines of the aforementioned local area to be repaired, generate 3D mesh models in both PLY and OBJ formats.

[0012] Furthermore, the specific steps of step S1 are as follows:

[0013] a. In the point cloud of the vertices of the grid model of the city wall ruins, use CloudCompare software to segment out the vertices of the area to be repaired;

[0014] b. Based on the vertices of the region to be repaired, calculate the corresponding bounding box and build the corresponding KD tree;

[0015] c. For all triangular meshes in the original OBJ model of the city wall ruins, determine whether they are within the above spatial bounding box; if they are within the above spatial bounding box, use the KD tree of the vertices of the region to be repaired to determine whether the vertices of the above triangular meshes are vertices of the region to be repaired; if so, select the above triangular meshes.

[0016] d. Based on the selected triangular mesh, generate a mesh model of the area to be repaired, including texture coordinate files and corresponding texture images.

[0017] Furthermore, the specific steps of step S2 are as follows:

[0018] a. Use the mesh model of the area to be repaired as the current mesh model;

[0019] b. In the current mesh model, each vertex is connected by edges, and a set A is established with the vertex and its corresponding multiple connection points as elements;

[0020] c. Based on the above set A, establish set B with each edge and its corresponding grid as elements;

[0021] d. Based on the above set B, select the grid corresponding to the edge whose length exceeds the threshold and subdivide it. Take the midpoint of the edge as the new vertex, and divide the grid with the above edge as the edge into two new grids. Delete the original grid.

[0022] e. Determine whether the mesh subdivision has been completed. If it has, convert the mesh model of the area to be repaired into a dense mesh model; otherwise, proceed to step b.

[0023] Furthermore, the specific steps of step S3 are as follows:

[0024] a. The points of the center point cloud of the area to be repaired correspond one-to-one with the triangular mesh of the dense mesh model; the dense mesh model adopts the PLY format, and the color values ​​of its vertices and the center point of the mesh are obtained by interpolation calculation in the texture image exported above through the corresponding texture coordinates; the software interface displays two windows, left and right, and both windows have the center point cloud of the area to be repaired; initially, one window has the dense mesh of the area to be repaired, and the other window does not have the dense mesh of the area to be repaired.

[0025] b. Point selection callback function: Deletes a triangular grid in one window, and the corresponding deleted grid is added to another window; includes non-connected deletion mode, connected deletion mode, and line deletion mode.

[0026] c. Perform the above operations interactively. Based on the connection deletion mode, delete the dense mesh of the connected line region in the region to be repaired, generating one or more connected mesh regions between the dense brick model and the local region to be repaired; obtain k dense brick models, the dense brick mesh and its index and the index of the corresponding original mesh, add them to the brick library and save them; obtain the center point cloud and the index of the local region to be repaired corresponding to the j deleted mesh connected regions and save them.

[0027] Furthermore, the specific steps of step S4 are as follows:

[0028] a. Based on the geometric characteristics of the local area to be repaired, the initial number of brick layers in the local area to be repaired is set to n; the center point cloud of the local area to be repaired has connectivity.

[0029] b. Based on the PCL point cloud library, create a point cloud segmentation object, set the straight line mode, and extract the interior points of the straight line region. That is, the straight line dense point cloud is saved as an element in an array. This process is repeated n times to obtain an array containing n dense point clouds. Sort the n dense point clouds in descending order of their center coordinate Y-values, with the positive Y-axis direction (width direction) upwards. Sort the n dense point clouds by their Y-values ​​and indices. After each extraction of the straight line dense point cloud, delete the dense point cloud from the center point cloud of the area to be repaired. Using the center line of the dense point cloud as a reference, with the center line passing through the center of the dense point cloud and its direction being the length direction (X-axis) and the Y-axis (width direction), delete points with a certain width (half the width of the dense point cloud) from the center point cloud of the repair area.

[0030] c. Based on the connectivity principle described above, obtain m sparse center point clouds that are adjacent to the dense center point cloud after performing step b.

[0031] d. For the above m sparse point clouds, m ≥ n, sort them in descending order of the number of point clouds, take the (n - 1) sparse point clouds with the largest number of point clouds, sort them in descending order of the Y-coordinate of the center of the (n - 1) sparse point clouds, with the positive direction of the Y-axis (i.e. the width direction) upward, and sort the Y-coordinate of the center of the (n - 1) sparse point clouds together with their index.

[0032] e. If the above-mentioned dense point cloud or sparse point cloud does not conform to the geometric features of the local area to be repaired, then interactively merge or delete one or more of the above-mentioned dense point cloud or sparse point cloud to obtain n1 dense point clouds and (n1 - 1) sparse point clouds (n ≥ n1), so that they conform to the geometric features of the local area to be repaired.

[0033] f. For the above (n1 - 1) sparse point clouds, perform principal component analysis (PCA) on the sparse point clouds based on the PCL point cloud library to obtain the X, Y, and Z axis vectors. Set the positive direction of the axial quantity of the first sparse point cloud, with its Y-axis (i.e., the positive direction of the width direction) pointing upwards. Using the X, Y, and Z axis directions of the first sparse point cloud as a reference, adjust the axis vectors of the other sparse point clouds so that the positive direction of their axial quantities is consistent with the axis vector of the first sparse point cloud.

[0034] g. Through the center line of the center point of each linear dense point cloud, offset by half the width of the dense point cloud along the Y-axis of the adjacent sparse point cloud, i.e., in the opposite direction of the width direction, to obtain the basic position line of the city bricks corresponding to the dense point cloud.

[0035] Furthermore, the specific steps of step S5 are as follows:

[0036] a. If the position lines of the edge part of the local area to be repaired are not on the same straight line as the basic position lines of the city bricks, it is necessary to interactively add, delete or modify the position lines of the city bricks and modify the corresponding basic position lines of the city bricks, i.e., execute step b.

[0037] b. After inputting the corresponding numbers via operation commands, the software enters the state of executing the corresponding function; then, the operation commands execute the following commands:

[0038] (1) In the center point cloud of the edge part of the local area to be repaired, the endpoints of the edge position center line are set by the operation command to generate the corresponding center line;

[0039] (2) Delete the set edge position center line and endpoints;

[0040] (3) Match the set edge position center line with the corresponding basic position center line, that is, shorten the length of the original basic position center line and the adjacent sparse point cloud so that the projection of the edge position center line on the width direction of the adjacent sparse point cloud does not overlap; segment the point cloud near the edge position center line, perform principal component analysis (PCA) on the corresponding dense point cloud, and calculate the length direction vector and the corresponding length of the dense point cloud corresponding to the edge position center line.

[0041] (4) Delete the set edge position lines and endpoints;

[0042] (5) Let X1 be the vector of the length direction of the dense point cloud corresponding to the center line, Z be the eigenvector corresponding to the minimum eigenvalue of the sparse point cloud of the basic position center line, and Z be the eigenvector corresponding to the minimum eigenvalue of the sparse point cloud of the basic position center line of the edge position center line; calculate the vector X of the position line direction corresponding to the center line using equation (1); calculate the vector Y of the width direction of the center line using equation (2), and make the positive direction of Y upward. The center line is offset along the opposite direction of Y by half the width of the dense point cloud corresponding to it to obtain the corresponding position line;

[0043] (1)

[0044] (2)

[0045] (6) Generate a brick mesh model of the local area to be repaired based on the edge position lines and the basic position lines;

[0046] (7) Save the edge position center line and the basic position center line to the file;

[0047] (8) Read in the file containing the saved location center line.

[0048] Furthermore, in step S6, the specific steps are as follows:

[0049] a. Select one or more dense brick grids in the brick library above. The grids are in PLY format. The edges of the bricks are stretched and trimmed. The dense brick grids are simplified to the original number of grids. The corresponding brick OBJ grid models are generated. The grids in a certain width area between the brick PLY and OBJ grid models are darkened to generate brick seam PLY and OBJ grid models.

[0050] b. The number of brick layers in the local area to be repaired is n1. Each layer has 1 basic position line and 0 or more edge position lines. The half of the adjacent position lines is extended by more than 1 times to overlap and merge. The position lines of each layer of bricks are arranged in order of length direction, and the position and model calculation of bricks and brick joints are performed in sequence. Let the current layer be the first layer and the current position line be the starting position line of the current layer.

[0051] c. The brick and joint models of the current position line of the current layer are calculated sequentially; when there is an overlap in the half of the position line, the distance of the current brick from the center of the adjacent position line is used as the weight to perform a fusion calculation of the position and length direction vector of the brick and joint, and generate the corresponding mesh model of the brick and joint.

[0052] d. Once the brick and joint models for all position lines in the current layer have been generated, proceed to step e if complete; otherwise, the current position line becomes the next position line in the current layer, and proceed to step c.

[0053] e. Once all layers of the local area to be repaired have been processed, if completed, a 3D model of the local area to be repaired in both PLY and OBJ formats will be generated; otherwise, the next layer will be taken as the current layer, the current position line will be taken as the starting position line of the current layer, and the process will proceed to step c.

[0054] Furthermore, the center point cloud of the local area to be repaired has connectivity: the points of the center point cloud of the repaired area correspond one-to-one with the triangular meshes in the dense mesh model, and the meshes have two common points, i.e., they are connected. Then the center points corresponding to the meshes in the center point cloud are connected. On the other hand, the two center points in the center point cloud correspond to two unconnected modules in the OBJ module, i.e., the two center points cannot achieve connectivity by sharing vertices through the corresponding meshes, but the distance between the two center points is less than the resolution of the center point cloud. Then the two center points are connected.

[0055] The non-connected deletion mode: deletes m grids from the dense grid at the selected location in one window, and adds the deleted m grids to another window;

[0056] The connectivity deletion mode is as follows: any two triangular meshes in a dense mesh have two common vertices (if any two triangular meshes in a dense mesh have two common vertices, then these two meshes are connected); in one window, the selected meshes are connected to their neighboring meshes to form a local mesh connectivity region, and then the mesh connectivity region is deleted; in another window, the deleted mesh connectivity region is added; if the repair software is in connectivity deletion mode and is in the state of saving the dense brick model or the center point cloud of the local area to be repaired, then the deleted mesh region or center point cloud is saved to the file according to the set file name; based on the connectivity of the deleted mesh region, a brick has at least one dense mesh connectivity region, and the dense mesh, its index, and the index of its corresponding original mesh are saved to the corresponding number of brick files;

[0057] e. The connection deletion mode: In the above non-connected deletion mode, in one window, n deletion operations are performed. Each time, m grids in the dense grid at the selected location are deleted. The center of the center point of the m grids is calculated. The n centers are used as the n endpoints of the line segment to form (n - 1) line segments. The grids in the cylinder with the above line segments as the central axis and radius r are deleted. The deleted grids are added in another window.

[0058] Furthermore, in step b of step S3, if the non-connection deletion mode is inverted, i.e., from 1 to 0, or from 0 to 1; 0 represents the connection deletion mode, and each time the center point of the deleted grid in the above non-connection deletion mode is saved as the endpoint of the line segment in the connection deletion mode; 1 represents the non-connection deletion mode; if the non-connection deletion mode is inverted, i.e., from 1 to 0, or from 0 to 1; 1 represents the non-connection deletion mode, and 0 represents the connected deletion mode; at the same time, the above connection deletion mode operation is performed as needed; finally, the state of saving the dense grid model of the city bricks or the center point cloud of the local area to be repaired is inverted, i.e., from 1 to 0, or from 0 to 1; 1 represents the state of saving the grid model or center point cloud, and the file name of the saved grid model or center point cloud is set; 0 represents the state of not saving the grid model or center point cloud.

[0059] Compared with the prior art, the beneficial effects of the present invention are:

[0060] This method produces 3D restoration models of city wall ruins with higher geometric precision, clearer textures, and greater realism. It can present the structure of the brick-clad walls more clearly, completely, and intuitively, providing an important basis for the subsequent protection and display of city wall ruins. At the same time, it has universality and can be applied to the restoration of other 3D mesh models with similar geometric features. Attached Figure Description

[0061] Figure 1 This is a flowchart illustrating the restoration method for a three-dimensional model of a city wall ruin according to the present invention.

[0062] Figure 2 This is a three-dimensional mesh model of the Iron Walled City West Gate site in an embodiment of the present invention;

[0063] Figure 3 This is a partial view of the three-dimensional mesh model of the Iron Walled City West Gate site in an embodiment of the present invention;

[0064] Figure 4 This is a vertex image extracted from the original OBJ mesh model of the Iron Walled City ruins in this embodiment of the invention;

[0065] Figure 5 This is a mesh model of the area to be restored extracted from the original OBJ model of the site in this embodiment of the invention;

[0066] Figure 6 This is a mesh model extracted based on vertices of the region to be repaired in this embodiment of the invention;

[0067] Figure 7 This is a mesh subdivision map of the region model to be repaired in an embodiment of the present invention;

[0068] Figure 8 The images show the original mesh model (left) and the generated dense mesh model (right) of the area to be repaired in this embodiment of the invention.

[0069] Figure 9 The image shows the dense triangular mesh model of the archaeological site (left) and the corresponding center point cloud (right) in an embodiment of the present invention.

[0070] Figure 10 The diagram shows the deletion of dense grids at the seven endpoints (left) and the addition of dense grids at the corresponding endpoints (right) in the connection deletion mode of this embodiment of the invention.

[0071] Figure 11 This is a diagram showing the deletion of dense grids (left) and the corresponding addition of dense grids (right) of 6 line segments in the connection deletion mode of this invention embodiment;

[0072] Figure 12 This is a diagram showing the deletion of dense grids (left) and the corresponding addition of dense grids (right) of 8 line segments in the connection deletion mode of this invention embodiment;

[0073] Figure 13 The diagram shows the deletion of connected region grids (left) and the corresponding addition of connected region grids (right) in the connected region deletion mode of this embodiment of the invention.

[0074] Figure 14 The diagram shows the deletion of connected region grids (right) and the corresponding addition of connected region grids (left) in the connected region deletion mode of this embodiment of the invention.

[0075] Figure 15Nine dense grid models of city bricks were extracted from the dense grid model of the Iron Walled City site in this embodiment of the invention;

[0076] Figure 16 This is a diagram showing the deletion of dense grids (left) and the corresponding addition of dense grids (right) of 6 line segments in the connection deletion mode of this invention embodiment;

[0077] Figure 17 The diagram shows the deletion of a connected region grid (left) and the addition of a corresponding connected region grid (right) in the connected deletion mode of this invention embodiment;

[0078] Figure 18 This is a diagram showing the deletion of another connected region grid (left) and the addition of the corresponding connected region grid (right) in the connected deletion mode of this embodiment of the invention;

[0079] Figure 19 The diagram shows the deletion of a connected region grid (right) and the addition of a corresponding connected region grid (left) in the connected deletion mode of this embodiment of the invention.

[0080] Figure 20 The diagram shows the deletion of another connected region grid (right) and the addition of the corresponding connected region grid (left) in the connected deletion mode of this embodiment of the invention.

[0081] Figure 21 The images shown are of the local area to be repaired (top) and two (middle and bottom) of the corresponding 12 grid connected areas in the dense grid model of the Tiewengcheng site in this embodiment of the invention.

[0082] Figure 22 This is a center point cloud (blue) map of the local area to be restored at the Iron Walled City site in an embodiment of the present invention;

[0083] Figure 23 This is a center point cloud image of the local area to be repaired before (left) and after (right) point cloud segmentation in an embodiment of the present invention;

[0084] Figure 24 This is the center point cloud image of the local area to be repaired after point cloud segmentation in this embodiment of the invention;

[0085] Figure 25 This is an adjusted image of the center point cloud of the local area to be repaired after point cloud segmentation in this embodiment of the invention.

[0086] Figure 26 This is a diagram showing the calculation (left) and adjustment (right) of the basic position lines of the city bricks in the local area to be repaired in an embodiment of the present invention.

[0087] Figure 27 This is a diagram showing the basic centerline of the bricks in the local area to be repaired in this embodiment of the invention.

[0088] Figure 28 This is a diagram showing the center lines (left and right) of the basic positions of the bricks in the local area to be repaired and the center line (right) of the edge positions in an embodiment of the present invention.

[0089] Figure 29 This is a diagram showing the basic position lines and edge position lines of the bricks in the local area to be repaired in an embodiment of the present invention;

[0090] Figure 30 This is a model of bricks and brick joints generated based on the location lines of the local area to be repaired in an embodiment of the present invention.

[0091] Figure 31 The figures above show the dense grid model of city bricks and the OBJ grid model of city bricks in the embodiments of the present invention.

[0092] Figure 32 The images show the original grid model (top) and the restored grid model (bottom) of the local area to be restored at the Iron Walled City site in this embodiment of the invention.

[0093] Figure 33 The images show the original grid model (top) and the restored grid model (middle and bottom) of the local area to be restored at the Iron Walled City site in this embodiment of the invention. Detailed Implementation

[0094] The method for restoring the three-dimensional model of the city wall ruins of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0095] Please see the appendix Figure 1-33 The embodiments and related experiments provided by this invention are as follows:

[0096] Example 1

[0097] The method for restoring a three-dimensional model of a city wall ruin according to an embodiment of the present invention includes the following steps:

[0098] S1: The original model of the city wall ruins is in OBJ format. The specific steps for separating the area to be restored from the original model are as follows:

[0099] a. In the point cloud of the vertices of the grid model of the city wall ruins, use CloudCompare software to segment out the vertices of the area to be repaired;

[0100] b. Based on the vertices of the region to be repaired, calculate the corresponding spatial bounding box (i.e., calculate the minimum and maximum values ​​of all the vertices in the X, Y, and Z dimensions to determine a cuboid bounding box) and build the corresponding KD tree;

[0101] c. For all triangular meshes in the original OBJ model of the city wall ruins, determine whether they are within the aforementioned spatial bounding box; if so, use the KD tree of the vertices of the region to be repaired to determine whether the vertices of the aforementioned triangular meshes are vertices of the region to be repaired; if so, select the aforementioned triangular meshes.

[0102] d. Based on the selected triangular mesh, generate a mesh model of the area to be repaired, including texture coordinate files and corresponding texture images.

[0103] S2: The above-mentioned model of the area to be repaired is subdivided into meshes, and the specific steps are as follows:

[0104] a. Use the mesh model of the area to be repaired as the current mesh model;

[0105] b. In the current mesh model, each vertex is connected by edges, and a set A is established with the vertex and its corresponding multiple connection points as elements;

[0106] c. Based on the above set A, establish set B with each edge and its corresponding grid as elements;

[0107] d. Based on the above set B, select the grid corresponding to the edge whose length exceeds the threshold and subdivide it. Take the midpoint of the edge as the new vertex, and divide the grid with the above edge as the edge into two new grids. Delete the original grid.

[0108] e. Determine if the mesh subdivision count has been completed. If it has, convert the mesh model of the area to be repaired into a dense mesh model; otherwise, proceed to step b.

[0109] S3: In the above model of the area to be repaired, interactively extract the dense brick model, add it to the brick library and save it; interactively extract the center point cloud of the local area to be repaired and save it. The specific steps are as follows:

[0110] a. The dense mesh model of the area to be repaired, referred to as the dense mesh model, has a point cloud formed by the center points of each mesh, referred to as the central point cloud. That is, the points of the central point cloud correspond one-to-one with the triangular meshes of the dense mesh model. The dense mesh model adopts the PLY format, and the color values ​​of its vertices and the center points of the meshes are obtained by interpolation calculation in the exported texture image using the corresponding texture coordinates. (That is, in the exported texture image, in the mesh of the 3D model before the mesh subdivision of the vertices and the center points, the texture coordinates of the vertices and the center points are obtained by linear interpolation calculation using the coordinates of the vertices, the center points, and the vertices of the meshes, as well as the texture coordinates of the vertices of the meshes.)

[0111] The software interface displays two windows, left and right, each containing the center point cloud of the area to be repaired. Initially, the left window shows a dense grid of the area to be repaired, while the right window does not.

[0112] b. Shift + Left Mouse Button Selection Callback Function: Deletes a triangular grid in the left (or right) window, and adds the deleted grid to the right (or left) window; includes non-connected deletion mode, connected deletion mode, and line deletion mode.

[0113] c. Non-connected deletion mode: Deletes m grids from the dense grid at the mouse click position in the left (right) window, and adds the deleted m grids to the right (left) window.

[0114] d. Connectivity Deletion Mode: Any two triangular meshes in a dense mesh share two common vertices, meaning they are connected. In the left (right) window, clicking on a mesh selects neighboring meshes in a connected manner, forming a local mesh connectivity region, which is then deleted. The deleted mesh connectivity region is added back in the right (left) window. If the repair software is in connectivity deletion mode and is simultaneously saving the dense brick model or the center point cloud of the local area to be repaired, the deleted mesh region or center point cloud will be saved to a file according to the set filename. Based on the connectivity of the deleted mesh regions, a brick has one dense mesh connected region. The dense mesh, its index, and the index of its corresponding original mesh are saved to one brick file. If a brick has two deleted mesh connected regions, it is because the two regions are not connected, corresponding to two modules of the OBJ mesh model. The dense mesh, its index, and the index of its corresponding original mesh are saved to two brick sub-files, which are then merged into one brick file. If a brick has more than two deleted mesh connected regions, the processing is similar to the above. Similar to the above operation, if the local area to be repaired has j deleted mesh connected regions, the corresponding center point cloud and the index of its points are saved to a file.

[0115] e. Wire Deletion Mode: In the non-connected deletion mode described above, n deletion operations are performed in the left (right) window. Each time, m grids in the dense grid at the mouse click position are deleted. The center of the center point of the m grids is calculated, and the n centers are used as the n endpoints of the line segment to form (n - 1) line segments. The grids in the cylinder with radius r as the central axis of the line segment are deleted. The deleted grids are added in the right (left) window.

[0116] f. The main functions of the keyboard event callback function include:

[0117] If the key is 'V', the non-connected deletion mode is inverted, that is, from 1 to 0, or from 0 to 1; 0 is the connected deletion mode, which saves the center of the center point of the deleted grid in the above non-connected deletion mode as the endpoint of the line segment in the connected deletion mode each time; 1 is the non-connected deletion mode.

[0118] If the key is 'L', the non-connected deletion mode is inverted, that is, it changes from 1 to 0, or from 0 to 1; 1 is the non-connected deletion mode, and 0 is the connected deletion mode.

[0119] If the key is 'P', the above connection deletion mode operation will be performed.

[0120] If the key is 'W', the state of saving the dense grid model of the city bricks or the center point cloud of the local area to be repaired is inverted, that is, from 1 to 0, or from 0 to 1; 1 means saving the grid model or center point cloud and setting the file name of the saved grid model or center point cloud; 0 means not saving the grid model or center point cloud.

[0121] g. Perform the above operations interactively. Based on the connection deletion mode, delete the dense mesh of the connected line region in the region to be repaired, generating one or more connected mesh regions between the brick dense model and the local region to be repaired; obtain k brick dense models, brick dense meshes and their indices and the indices of their corresponding original meshes, add them to the brick library and save them; obtain the center point cloud and the indices of the local regions to be repaired corresponding to the j deleted mesh connected regions and save them.

[0122] S4: Based on the dense point cloud segmentation of the central point cloud of the local area to be repaired, calculate the basic position line of the city bricks. The specific steps are as follows:

[0123] a. Based on the geometric characteristics of the local area to be repaired, the initial number of brick layers in the local area to be repaired is set to n. The connectivity principle of the central point cloud of the local area to be repaired is as follows: the points of the central point cloud of the repaired area correspond one-to-one with the triangular meshes in the dense mesh model. If there are two common points between the meshes, they are connected. Then the central points corresponding to the meshes in the central point cloud are connected. On the other hand, if two central points in the central point cloud correspond to two unconnected modules in the OBJ module, that is, the two central points cannot be connected by sharing vertices through the corresponding meshes, but the distance between the two central points is less than the resolution of the central point cloud, then the two central points are connected.

[0124] b. Based on the PCL point cloud library, create a point cloud segmentation object, set it to straight line mode, and extract the interior points of the straight-line region. That is, save the straight-line dense point cloud as an element in an array. Repeat this process n times to obtain an array containing n dense point clouds. Sort the n dense point clouds in descending order of their center Y-coordinates, with the positive Y-axis (width direction) upwards. Then, sort the n dense point clouds by their Y-values ​​and indices. After each extraction of the straight-line dense point cloud, remove it from the center point cloud of the region to be repaired. Using the centerline of the dense point cloud as a reference, with the centerline passing through the center of the dense point cloud and its direction being the length direction (X-axis) and the Y-axis (width direction), remove points of a certain width from the center point cloud of the region to be repaired. This operation ensures that sparse point clouds adjacent to the straight-line dense point cloud are effectively segmented.

[0125] c. After performing step b, based on the above connectivity principle, obtain m sparse center point clouds that are adjacent to the dense center point cloud.

[0126] d. For the above m sparse point clouds, m ≥ n, sort them in descending order of the number of point clouds, take the (n - 1) sparse point clouds with the largest number of point clouds, and sort them in descending order of the Y-coordinate of the center of the (n - 1) sparse point clouds, with the positive direction of the Y-axis (i.e. the width direction) upward. Sort the Y-coordinate of the center of the (n - 1) sparse point clouds together with their index.

[0127] e. If the above-mentioned dense point cloud or sparse point cloud does not conform to the geometric features of the local area to be repaired, then interactively merge or delete one or more of the above-mentioned dense point cloud or sparse point cloud to obtain n1 dense point clouds and (n1 - 1) sparse point clouds (n ≥ n1), so that they conform to the geometric features of the local area to be repaired.

[0128] f. For the above (n1 - 1) sparse point clouds, perform principal component analysis (PCA) on the sparse point clouds based on the PCL point cloud library to obtain the X, Y, and Z axis vectors. Set the positive direction of the axial quantity of the first sparse point cloud, with its Y-axis, i.e., the positive direction of the width direction, pointing upwards. Using the X, Y, and Z axis directions of the first sparse point cloud as a reference, adjust the axis vectors of the other sparse point clouds so that the positive direction of their axial quantities is consistent with the axis vector of the first sparse point cloud.

[0129] g. Through the center line of the center point of each linear dense point cloud, offset by half the width of the dense point cloud along the opposite direction of the Y-axis (width direction) of the adjacent sparse point cloud to obtain the basic position line of the city bricks corresponding to the dense point cloud.

[0130] S5: If the edge of the area to be repaired deviates significantly from the basic position line of the city bricks, interactively add, delete, or modify the edge position line of the city bricks, and modify the corresponding basic position line of the city bricks. The specific steps are as follows:

[0131] a. In the local area to be repaired, a dense point cloud of straight lines is segmented out. If the position lines of the edge part of the local area to be repaired are not on the same straight line as the basic position lines of the city bricks, it is necessary to interactively add, delete or modify the position lines of the city bricks and modify the corresponding basic position lines of the city bricks, that is, to execute step b.

[0132] b. Keyboard event callback + (Shift + left mouse button selection callback) combination to execute the corresponding operation. When the user presses 'V' on the keyboard, they are prompted to enter a number. After entering the corresponding number, the software enters the state to execute the corresponding function; then, holding down the Shift key and clicking the left mouse button executes the corresponding function. The main functions are as follows:

[0133] '1': In the center point cloud of the edge part of the local area to be repaired, use Shift + left mouse button to set the endpoint of the edge position center line and generate the corresponding center line.

[0134] '2': Delete the set edge position center line and endpoints.

[0135] '3': Match the set edge position centerline with the corresponding basic position centerline, that is, shorten the length of the original basic position centerline and the adjacent sparse point cloud so that their projections in the width direction of the adjacent sparse point cloud do not overlap with the edge position centerline; segment the point cloud near the edge position centerline, perform principal component analysis (PCA) on the corresponding dense point cloud, and calculate the length direction vector and the corresponding length of the dense point cloud corresponding to the edge position centerline.

[0136] '4': Delete the set edge position lines and endpoints.

[0137] '5': Let X1 be the vector of the length direction of the dense point cloud corresponding to the center line, Z be the eigenvector corresponding to the minimum eigenvalue of the sparse point cloud of the basic position center line, and Z be the eigenvector corresponding to the minimum eigenvalue of the sparse point cloud of the neighboring basic position center line; calculate the vector X of the position line direction corresponding to the center line using equation (1); calculate the vector Y of the width direction of the center line using equation (2), and make the positive direction of Y upward. The center line is offset along the opposite direction of Y by half the width of the dense point cloud corresponding to it to obtain the corresponding position line.

[0138] (1)

[0139] (2)

[0140] '6': Generate a brick mesh model of the local area to be repaired based on the edge location line and the basic location line.

[0141] '8': Save the edge position center line and the basic position center line to the file.

[0142] '9': Read in the file containing the saved center line location.

[0143] S6: Based on the brick location lines of the aforementioned local area to be repaired, generate 3D mesh models in both PLY and OBJ formats. The specific steps (i.e., the function corresponding to the number '6' in the above keyboard event callback + (Shift + left mouse button selection callback) combination method) are as follows:

[0144] a. Select one or more dense brick grids in the brick library above. The grids are in PLY format. The edges of the bricks are stretched and trimmed. The dense brick grids are simplified to the original number of grids. The corresponding brick OBJ grid models are generated. The grids in a certain width area between the brick PLY and OBJ grid models are darkened to generate brick seam PLY and OBJ grid models.

[0145] b. The number of brick layers in the local area to be repaired is n1. Each layer has 1 basic position line and 0 or more edge position lines. The adjacent position lines are extended by more than 1 times to overlap and merge. The position lines of each layer are arranged in order of length direction, and the position and model calculation of bricks and brick joints are performed in sequence. Let the current layer be the first layer and the current position line be the starting position line of the current layer.

[0146] c. The brick and joint models of the current position line of the current layer are calculated sequentially; when there is an overlap in the half of the position line, the distance of the current brick from the center of the adjacent position line is used as the weight to perform a fusion calculation of the position and length direction vector of the brick and joint, and generate the corresponding mesh model of the brick and joint.

[0147] d. Have all the brick and joint models for the current layer been generated? If so, proceed to step e; otherwise, the current position line is the next position line for the current layer, and proceed to step c.

[0148] e. Has all layers of the local area to be repaired been processed? If so, generate a 3D model of the local area to be repaired in both PLY and OBJ formats; otherwise, take the next layer as the current layer, the current position line as the starting position line of the current layer, and proceed to step c.

[0149] Experiment 1

[0150] 1) Experimental framework of the present invention

[0151] This work, based on the PCL point cloud library, researched and developed 3D mesh model restoration software for the Tiewengcheng site. The overall framework of the related work is as follows: Figure 1 As shown.

[0152] By using DJI drones to conduct oblique photogrammetry of the West Gate site of Tiewengcheng, a corresponding 3D model was generated. Through human-computer interaction and program implementation, the spatial location and grid model of the city bricks in the part of the site to be restored were determined, and a city brick model of the area to be restored was generated, thus realizing digital restoration.

[0153] 2) 3D modeling of the ruins of the west gate of the Iron Walled City

[0154] like Figure 1 The overall framework of this invention involves control point measurement of the West Gate site of Tiewengcheng, oblique photogrammetry using DJI drones, planning and executing a combined high- and low-altitude flight path for the site, and obtaining approximately 1387 high-resolution photos. A three-dimensional mesh model in OSGB format is generated using the Smart3D software and then converted into a three-dimensional mesh model in OBJ format using the OSGBLab software.

[0155] The repair software of this invention repairs 3D models in OBJ format. Figure 2 The displayed image is a 3D mesh model (OBJ format) of the site of the west gate of the Iron Walled City. Figure 3 The image shows a city wall located in a low-lying area, composed of several layers of bricks. It has turned black due to long-term immersion in groundwater, resulting in poor 3D reconstruction and necessitating digital restoration. This relates to the research and development of the restoration software for the Iron Walled City site as described in this invention.

[0156] 3) Separate the areas to be restored from the original model of the Iron Walled City ruins.

[0157] The original 3D mesh model (OBJ format) of the Iron Walled City West Gate site has 33,303,288 vertices and 37,944,612 texture vertices. Using the point cloud segmentation function of CloudCompare software, 224,192 vertices were extracted from the point cloud corresponding to the vertices of the original mesh model to extract the vertices of the area to be repaired. Figure 4 ).

[0158] A partial point cloud extracted from the point cloud of the vertices of the original OBJ model based on the Iron Walled City West Gate site. Figure 4 ), in the original OBJ model ( Figure 2 Based on this, through the following algorithm ( Figure 5 Extract the mesh model corresponding to the aforementioned local point cloud. Figure 6 The number of vertices is 224,192, the number of triangular meshes is 427,293, the corresponding number of texture vertices is 254,482, and the number of texture images is 6.

[0159] During the data processing, the repair software uses the PLY format for the network model of the area to be repaired. The elements include: vertices, which store the three-dimensional coordinates and color attributes; and triangular meshes (faces), which store the vertex indices of the triangular meshes.

[0160] 4) Repair the mesh subdivision of the region model

[0161] The algorithm involves subdividing the model of the region to be repaired (PLY format) into a dense mesh model. The main process of the algorithm is as follows: Figure 7 As shown.

[0162] like Figure 8 As shown, the original mesh model of the area to be repaired has 224,192 vertices and 427,293 meshes. After 5 iterations, the dense mesh model generated for the area to be repaired has 1,916,883 vertices and 3,798,182 meshes, corresponding to 1,947,173 texture vertices. The mesh index of the original mesh model corresponding to the dense mesh is saved so that the mesh number of the dense brick model can be simplified to the mesh number of the corresponding original brick model later.

[0163] 5) Extraction of dense brick model and local area to be repaired

[0164] In the previous section, the original model (PLY format) of the area to be repaired was subdivided into a dense mesh model. Within this dense mesh model, the dense brick model and the local area to be repaired were extracted through a combination of human-computer interaction and programmatic implementation. The specific steps included:

[0165] (1) The dense grid model of the city wall to be restored at the west gate of the Iron Walled City site (hereinafter referred to as the dense grid model) is formed by the center point of each grid, which is called the central point cloud. That is, the points of the central point cloud correspond one-to-one with the triangular grids of the dense grid model. The dense grid model adopts the PLY format, and its vertex color value is calculated by interpolation in the texture image exported above through the texture coordinates of the vertex.

[0166] Repair the left and right windows of the software, such as Figure 9-14 As shown in Figures 16-20, the dark color represents the triangular mesh, and the light color represents the center point of the corresponding mesh. Both windows have a center point cloud. Initially, the wall repair area in the left window has a dense mesh, while the right window does not. Figure 9 ).

[0167] (2) Shift + Left Mouse Button Selection Callback Function

[0168] like Figure 9-14As shown in Figures 16-20, deleting a triangular mesh in the left (or right) window will add the corresponding deleted mesh to the right (or left) window. The repair software offers two methods for deleting meshes:

[0169] Non-connected deletion mode: Deletes m grids from the dense grid at the mouse click location in the left (right) window, and adds the deleted m grids to the right (left) window. Figure 10 ).

[0170] Connected deletion mode: In a dense mesh, any two triangular meshes share two common vertices, meaning they are connected. In the left (right) window, the mesh at the mouse click location is selected as a connected neighboring mesh, forming a local mesh region, which is then deleted. The deleted mesh region is added back to the right (left) window. Figure 12-14 16-20).

[0171] If the repair software is in connectivity deletion mode and is simultaneously saving the dense brick model or the center point cloud of the local area to be repaired, then the deleted mesh region will be saved to a file according to the set filename. Based on the connectivity of the deleted mesh region, Figure 13 (Right) In the middle, a deleted mesh connected region is saved to a brick file; Figure 18-20 In the right image, two deleted mesh connected regions of a brick (the two regions are not connected because they correspond to two modules of the OBJ mesh model) are saved to two brick sub-files and then merged into one brick file; Figure 21-22 In the process, the center point cloud of the local region to be repaired corresponding to the 12 deleted grid connected regions and the index of its points (corresponding to the deleted dense grid) are saved to a file.

[0172] (3) The main function of the keyboard event callback function

[0173] Connected deletion mode: In the non-connected deletion mode described above, n deletion operations are performed in the left (right) window. Each time, the center points of m grids are deleted. Their corresponding centers are calculated, and these n centers are used as the n endpoints of a line segment, forming (n - 1) line segments. The grids within a cylinder of a certain radius r, with these line segments as the central axis, are deleted. The deleted grids are then added back to the right (left) window. Figure 10-12 16).

[0174] If the key is 'V', the non-connected deletion mode is inverted, that is, from 1 to 0, or from 0 to 1; 0 is the connected deletion mode, and each time the center of the deletion grid in the above non-connected deletion mode is saved as the endpoint of the line segment in the connected deletion mode; 1 is the non-connected deletion mode.

[0175] If the key is 'L', the non-connected deletion mode is inverted, that is, it changes from 1 to 0, or from 0 to 1; 1 is the above non-connected deletion mode; 0 is the connected deletion mode.

[0176] If the key is 'P', the above connection deletion mode operation will be performed.

[0177] If the key is 'W', the state of saving the dense grid model of the city bricks or the center point cloud of the local area to be repaired is inverted, that is, from 1 to 0, or from 0 to 1; 1 means saving the grid model or center point cloud and setting the file name of the saved grid model or center point cloud; 0 means not saving the grid model or center point cloud.

[0178] After the above operations, several dense brick models can be saved to corresponding files. For example... Figure 15 As shown, nine dense brick models were extracted and saved from the dense grid model of the area to be repaired.

[0179] In the dense mesh model of the area to be repaired, the process of extracting the center point cloud of the local area to be repaired is the same as the process of extracting the dense model of the city bricks. Using the connection deletion mode, several connected mesh regions without surrounding meshes are obtained (these mesh regions correspond to modules of different OBJ mesh models; their internal meshes are connected, but they are not connected to each other). The center point cloud corresponding to the mesh model of the repaired area is saved module by module. For example... Figure 21 As shown, the area within the white box in the top image represents the connected mesh regions to be deleted; there are a total of 12 connected regions. The yellow boxes in the middle and bottom images represent two of these 12 connected mesh regions. Figure 22 The center point cloud (blue) of the local area to be repaired corresponds to 12 grid connected regions. The corresponding 12 center point clouds of the local area to be repaired and their point indices (corresponding to the deleted dense grids) are saved to a file. In the next subsection, the saved center point clouds and their point indices are read into memory to calculate the brick position lines of the local area to be repaired.

[0180] 6) Calculation of the location line of the city bricks in the local area to be repaired

[0181] In the previous section, based on the connected deletion mode of the dense grid model of the Iron Walled City West Gate site, the center point cloud of the corresponding local area to be restored was obtained. Figure 22 The dense grid corresponds one-to-one with the center point. For example... Figure 23 As shown in (left), the blue area is the central point cloud. The geometric feature of the central point cloud is that it has several linear dense and sparse point cloud regions, corresponding to the position and axis vector of the city brick in the local area to be repaired.

[0182] The basic position lines of the city bricks are determined by the following steps:

[0183] (1) Point cloud segmentation and adjustment of the local area to be repaired

[0184] Set the initial number of brick layers n in the local area to be repaired, taking into account the geometric characteristics of the Iron Wall City ruins repair area ( Figure 23 (Left) The initial number of brick layers n in the repair area is set to 13.

[0185] Based on the PCL point cloud library, a point cloud segmentation object is created, and the straight line mode (pcl::SACMODEL_LINE) is set. The interior points of the straight line region are extracted, that is, the dense point cloud of the straight line is saved as an element and stored in an array. This process is repeated n times to obtain a dense point cloud containing n points. Figure 23 (right) and Figure 24 The array is sorted by the Y-values ​​and indexes of the n dense point clouds in descending order of the center coordinates of the n dense point clouds.

[0186] After each extraction of the linear dense point cloud, the dense point cloud is deleted from the center point cloud of the repair area; points with a certain width along the Y-axis (width direction) upwards and downwards are deleted from the center point cloud of the repair area, using the center line of the dense point cloud (passing through the center of the dense point cloud and with the direction as the X-axis) as the reference; the above operations ensure that the sparse point cloud adjacent to the linear dense point cloud is effectively separated. Figure 23 (right) and Figure 24 ).

[0187] The connectivity principle of the central point cloud of the local area to be repaired is as follows: if the points of the central point cloud of the repaired area correspond one-to-one with the triangular meshes in the dense mesh model, and there are two common points between the meshes (i.e., they are connected), then the corresponding points of the central point cloud are connected; at the same time, if two points in the central point cloud correspond to two unconnected modules in the OBJ module, but the distance between the two points is less than the resolution (point spacing) of the central point cloud, then these two points are connected. For the central point cloud generated after performing step (2), according to the above connectivity principle, m sparse central point clouds adjacent to the dense central point cloud are generated.

[0188] For the above m (m ≥ n) sparse point clouds, sort them in descending order according to the number of point clouds. Take the (n - 1) sparse point clouds with the most point clouds, and sort them in descending order according to the Y-value of the center coordinates of the (n - 1) sparse point clouds. Then sort the Y-value of the center of the (n - 1) sparse point clouds and the index.

[0189] Based on the results of the repair software execution, the above n dense point clouds and (n - 1) sparse point clouds are readjusted to obtain n1 dense point clouds and (n1 - 1) sparse point clouds (n ≥ n1). For example... Figure 23 (right) and Figure 24 As shown, the 4th and 5th dense point clouds are merged into one dense point cloud, and the 1st dense point cloud and its adjacent sparse point cloud are deleted. The adjusted result is as follows. Figure 25 As shown. At this time, n = 13, n1 = 11, that is, the center point cloud of the local area to be repaired originally had 13 dense point clouds in a straight line and 12 sparse point clouds. Figure 23 (Right)); After readjustment, it becomes 11 linear dense point clouds and 10 adjacent sparse point clouds ( Figure 25 ).

[0190] (2) After the point cloud of the central point cloud of the local area to be repaired is segmented and adjusted, the resulting (n1 - 1) sparse point clouds are subjected to principal component analysis (PCA) using the PCA class in the PCL library to obtain the X, Y and Z axis vectors. The positive direction of the axial quantity of the first sparse point cloud is set, and its positive Y axis direction is upward. Based on the X, Y and Z axis directions of the first sparse point cloud, the direction vectors of other sparse point clouds are adjusted so that the positive direction of their axial quantity is consistent with the axis vector of the first sparse point cloud.

[0191] By passing through the center line of the center point of each linear dense point cloud, and offsetting it in the opposite direction of the Y-axis of the adjacent sparse point cloud by half the width of the dense point cloud, the basic position line of the city brick corresponding to the dense point cloud is obtained. Figure 26 In the left image, the X, Y, and Z axes of the (n1-1) sparse point clouds are not aligned, causing the calculated basic position lines of the city bricks (red or black) to be above or below the corresponding dense point clouds. Figure 26 In the right image, the X, Y, and Z axes of the (n1 - 1) sparse point clouds are referenced to the first sparse point cloud, and the axes are aligned so that the basic position lines of the city bricks (red or black) are uniformly below the straight dense point cloud.

[0192] (3) The dense point cloud in the straight line segmented in the local area to be repaired has a large difference between the edge position of the local area and the actual position of the dense point cloud, which needs to be adjusted by manual interaction.

[0193] For the basic and edge center lines of the local area to be repaired, the center line passes through the center of the dense point cloud, and its direction is the length direction of the dense point cloud. The main steps for adding, deleting, and modifying the position lines of the local area to be repaired are as follows:

[0194] Load the center point cloud (blue) of the local area to be repaired; its linear dense point cloud has a center line ( Figure 27 , 28 (Left)).

[0195] The software uses a combination of keyboard event callbacks and (Shift + left mouse button selection callback) to execute corresponding operations. When the user presses 'V' on the keyboard, they are prompted to enter a number. After entering the number, the software enters the state to execute the corresponding function; then, holding down the Shift key and clicking the left mouse button executes the corresponding function. The main numeric functions are as follows:

[0196] '1': In the center point cloud of the edge portion of the local area to be repaired ( Figure 28 (Left) By using Shift + left mouse button, set the endpoints of the edge position center line to generate the corresponding center line. Figure 28 (right)).

[0197] '2': Delete the set edge position center line and endpoints.

[0198] '3': Match the set edge position centerline with the corresponding basic position centerline, that is, shorten the length of the original basic position centerline so that its projection on the width direction of the adjacent sparse point cloud does not overlap with the edge position centerline; segment the point cloud near the edge position centerline, perform principal component analysis (PCA) on the corresponding dense point cloud, and calculate the axis vector and corresponding length of the dense point cloud corresponding to the edge position centerline.

[0199] '4': Delete the set edge position lines and endpoints.

[0200] '5': Let X1 be the vector of the length direction of the dense point cloud corresponding to the center line, Z be the eigenvector corresponding to the minimum eigenvalue of the sparse point cloud of the basic position center line, and Z be the eigenvector corresponding to the minimum eigenvalue of the sparse point cloud of the basic position center line of the edge position center line; calculate the vector X of the position line direction corresponding to the center line using equation (1); calculate the vector Y of the width direction of the center line using equation (2), and make the positive direction of Y upward.

[0201] (1)

[0202] (2)

[0203] The center line is offset in the opposite direction of Y by half the width of the corresponding dense point cloud to obtain the corresponding position line. Figure 29 (A red or black line).

[0204] '6': Based on the edge location lines and basic location lines, generate the brick mesh model of the local area to be repaired (next part).

[0205] '8': Save the edge position center line and the basic position center line to the file.

[0206] '9': Read in the file containing the saved center line location.

[0207] 7) Brick restoration based on the location line of the restoration area

[0208] Based on the keyboard event callback + (Shift + left mouse button selection callback) combination method in the previous section, the function corresponding to the number '6' is used to reconstruct the brick model of the local area to be repaired based on the position fusion of the brick position lines. The main steps are shown in the flowchart below. Figure 30 As shown.

[0209] like Figure 31 As shown, based on the city brick library ( Figure 15 A dense grid model of a city brick (PLY format) is extracted, and the dense grid of the city brick is simplified to the number of original OBJ grid models to generate a corresponding simplified city brick grid model (PLY format), which greatly reduces the number of triangular grids. By adjusting the vertex coordinates of the city brick grid model, the city brick model is shortened and flattened in the horizontal direction and lengthened and flattened in the vertical direction, so that the edges of the brick model in the local area to be repaired are neat and there are no gaps between the city bricks in the vertical direction. Based on the simplified city brick grid model (PLY format), a city brick OBJ grid model with the same number of vertices and grids is generated.

[0210] In the above simplified brick mesh model (PLY format), a triangular mesh of a certain width in the middle is selected to generate the corresponding simplified brick mesh model (PLY format). The brick joint model is shortened and flattened in the horizontal direction and lengthened and flattened in the vertical direction, so that the edges of the brick joint model in the local area to be repaired are neat and there are no gaps between the vertical brick joints. Based on the simplified brick mesh model (PLY format), a brick joint OBJ mesh model with the same number of vertices and meshes is generated and its texture is darkened.

[0211] like Figure 31 As shown, the number of triangular meshes in the OBJ mesh model of the city brick is much less than that in the dense mesh model of the city brick (PLY format), but the texture details are clearer. This is because texture mapping makes the texture of the OBJ mesh model richer than that of the PLY mesh model, that is, the three-dimensional effect is more realistic.

[0212] like Figures 32-33As shown, the repair software has implemented the function of generating brick models of the local area to be repaired based on the location line. In the repair area of ​​the Iron Walled City, compared with the original OBJ 3D mesh model before repair, the structure of the brick wall of the repaired OBJ 3D mesh model is clearer, more complete and more intuitive.

[0213] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for restoring a three-dimensional model of a city wall ruin, characterized in that, Includes the following steps: S1: The original model of the city wall ruins is in OBJ format. The area to be restored is separated from the original model. S2: Subdivide the mesh of the above-mentioned area to be repaired; S3: In the above model of the area to be repaired, interactively extract the dense model of the city bricks, add it to the city brick library and save it; interactively extract the center point cloud of the local area to be repaired and save it. S4: Based on the dense point cloud segmentation of the central point cloud of the local area to be repaired, calculate the basic position line of the city bricks. S5: If the position line of the edge part of the local area to be repaired is not on the same straight line as the basic position line of the city brick, then interactively add, delete or modify the position line of the city brick edge, and modify the corresponding basic position line of the city brick. S6: Based on the brick location lines of the aforementioned local area to be repaired, generate 3D mesh models in both PLY and OBJ formats.

2. The method for restoring a three-dimensional model of a city wall ruin according to claim 1, characterized in that: The specific steps of step S1 are as follows: a. In the point cloud of the vertices of the grid model of the city wall ruins, use CloudCompare software to segment out the vertices of the area to be repaired; b. Based on the vertices of the region to be repaired, calculate the corresponding bounding box and build the corresponding KD tree; c. For all triangular meshes in the original OBJ model of the city wall ruins, determine whether they are within the aforementioned spatial bounding box; If it is within the bounding box of the space above, the KD tree of the vertices of the region to be repaired is used to determine whether the vertices of the above triangular mesh are vertices of the region to be repaired; if so, the above triangular mesh is selected. d. Based on the selected triangular mesh, generate a mesh model of the area to be repaired, including texture coordinate files and corresponding texture images.

3. The method for restoring a three-dimensional model of a city wall ruin according to claim 1 or 2, characterized in that: The specific steps of step S2 are as follows: a. Use the mesh model of the area to be repaired as the current mesh model; b. In the current mesh model, each vertex is connected by edges, and a set A is established with the vertex and its corresponding multiple connection points as elements; c. Based on the above set A, establish set B with each edge and its corresponding grid as elements; d. Based on the above set B, select the grid corresponding to the edge whose length exceeds the threshold and subdivide it. Take the midpoint of the edge as the new vertex, and divide the grid with the above edge as the edge into two new grids. Delete the original grid. e. Determine whether the mesh subdivision has been completed. If it has, convert the mesh model of the area to be repaired into a dense mesh model; otherwise, proceed to step b.

4. The method for restoring a three-dimensional model of a city wall ruin according to claim 3, characterized in that: The specific steps of step S3 are as follows: a. The points of the center point cloud of the area to be repaired correspond one-to-one with the triangular mesh of the dense mesh model; the dense mesh model adopts the PLY format, and the color values ​​of its vertices and the center point of the mesh are obtained by interpolation calculation in the texture image exported above through the corresponding texture coordinates; the software interface displays two windows, left and right, and both windows have the center point cloud of the area to be repaired; initially, one window has the dense mesh of the area to be repaired, and the other window does not have the dense mesh of the area to be repaired. b. Selection callback function: Deletes a triangular grid in one window, and the corresponding deleted grid is added to another window; This includes non-connected deletion mode, connected deletion mode, and connected deletion mode; c. Perform the above operations interactively. Based on the connection deletion mode, delete the dense mesh of the connected line region in the region to be repaired, generating one or more connected mesh regions between the dense brick model and the local region to be repaired; obtain k dense brick models, the dense brick mesh and its index and the index of the corresponding original mesh, add them to the brick library and save them; obtain the center point cloud and the index of the local region to be repaired corresponding to the j deleted mesh connected regions and save them.

5. The method for restoring a three-dimensional model of a city wall ruin according to claim 1, 2, or 4, characterized in that: The specific steps of step S4 are as follows: a. Based on the geometric features of the local area to be repaired, the initial number of brick layers in the local area to be repaired is set to n; the center point cloud of the local area to be repaired has connectivity; b. Based on the PCL point cloud library, create a point cloud segmentation object, set the straight line mode, and extract the interior points of the straight line region. That is, the straight line dense point cloud is saved as an element in an array. This process is repeated n times to obtain an array containing n dense point clouds. Sort the n dense point clouds in descending order of their center coordinate Y-values, with the positive Y-axis direction (width direction) upwards. Sort the n dense point clouds by their Y-values ​​and indices. After each extraction of the straight line dense point cloud, delete the dense point cloud from the center point cloud of the area to be repaired. Using the centerline of the dense point cloud as a reference, with the centerline passing through the center of the dense point cloud and its direction being the length direction (X-axis) and the Y-axis (width direction), delete points with a certain width from the center point cloud of the repair area. c. Based on the connectivity principle described above, obtain m sparse center point clouds that are adjacent to the dense center point cloud after performing step b. d. For the above m sparse point clouds, m ≥ n, sort them in descending order of the number of point clouds, take the (n - 1) sparse point clouds with the largest number of point clouds, sort them in descending order of the Y-coordinate of the center of the (n - 1) sparse point clouds, with the positive direction of the Y-axis (i.e. the width direction) upward, and sort the Y-coordinate of the center of the (n - 1) sparse point clouds together with their index. e. If the above-mentioned dense point cloud or sparse point cloud does not conform to the geometric features of the local area to be repaired, then interactively merge or delete one or more of the above-mentioned dense point cloud or sparse point cloud to obtain n1 dense point clouds and (n1 - 1) sparse point clouds (n ≥ n1), so that they conform to the geometric features of the local area to be repaired. f. For the above (n1 - 1) sparse point clouds, based on the PCL point cloud library, perform principal component analysis (PCA) on the sparse point clouds to obtain the X, Y and Z axis vectors. Set the positive direction of the axial quantity of the first sparse point cloud, and its Y axis, which is the positive direction of the width direction, is upward. Using the X, Y, and Z axes of the first sparse point cloud as a reference, the axis vectors of other sparse point clouds are adjusted so that the positive direction of their axis vectors is consistent with the axis vector of the first sparse point cloud. g. Through the center line of the center point of each linear dense point cloud, offset by half the width of the dense point cloud along the Y-axis of the adjacent sparse point cloud, i.e., in the opposite direction of the width direction, to obtain the basic position line of the city bricks corresponding to the dense point cloud.

6. The method for restoring a three-dimensional model of a city wall ruin according to claim 5, characterized in that: The specific steps of step S5 are as follows: a. If the position lines of the edge part of the local area to be repaired are not on the same straight line as the basic position lines of the city bricks, it is necessary to interactively add, delete or modify the position lines of the city bricks and modify the corresponding basic position lines of the city bricks, i.e., execute step b. b. After inputting the corresponding numbers via operation commands, the software enters the state of executing the corresponding function; then, the operation commands execute the following commands: (1) In the center point cloud of the edge part of the local area to be repaired, the endpoints of the edge position center line are set by the operation command to generate the corresponding center line; (2) Delete the set edge position center line and endpoints; (3) Match the set edge position center line with the corresponding basic position center line, that is, shorten the length of the original basic position center line and the adjacent sparse point cloud so that the projection of the edge position center line on the width direction of the adjacent sparse point cloud does not overlap; segment the point cloud near the edge position center line, perform principal component analysis (PCA) on the corresponding dense point cloud, and calculate the length direction vector and the corresponding length of the dense point cloud corresponding to the edge position center line. (4) Delete the set edge position lines and endpoints; (5) Let X1 be the vector of the length direction of the dense point cloud corresponding to the center line, Z be the eigenvector corresponding to the minimum eigenvalue of the sparse point cloud of the basic position center line, and Z be the eigenvector corresponding to the minimum eigenvalue of the sparse point cloud of the basic position center line of the edge position center line; calculate the vector X of the position line direction corresponding to the center line using equation (1); calculate the vector Y of the width direction of the center line using equation (2), and make the positive direction of Y upward. The center line is offset along the opposite direction of Y by half the width of the dense point cloud corresponding to it to obtain the corresponding position line; (1)； (2)； (6) Generate a brick mesh model of the local area to be repaired based on the edge position lines and the basic position lines; (7) Save the edge position center line and the basic position center line to the file; (8) Read in the file containing the saved location center line.

7. The method for restoring a three-dimensional model of a city wall ruin according to claim 1, 2, 4 or 6, characterized in that: The specific steps in step S6 are as follows: a. Select one or more dense brick grids in the brick library above. The grids are in PLY format. The edges of the bricks are stretched and trimmed. The dense brick grids are simplified to the original number of grids. The corresponding brick OBJ grid models are generated. The grids in a certain width area between the brick PLY and OBJ grid models are darkened to generate brick seam PLY and OBJ grid models. b. The number of brick layers in the local area to be repaired is n1. Each layer has 1 basic position line and 0 or more edge position lines. The half of the adjacent position lines is extended by more than 1 times to overlap and merge. The position lines of each layer of bricks are arranged in order of length direction, and the position and model calculation of bricks and brick joints are performed in sequence. Let the current layer be the first layer and the current position line be the starting position line of the current layer. c. The brick and joint models of the current position line of the current layer are calculated sequentially; when there is an overlap in the half of the position line, the distance of the current brick from the center of the adjacent position line is used as the weight to perform a fusion calculation of the position and length direction vector of the brick and joint, and generate the corresponding mesh model of the brick and joint. d. Once the brick and joint models for all position lines in the current layer have been generated, proceed to step e if complete; otherwise, the current position line becomes the next position line in the current layer, and proceed to step c. e. Once all layers of the local area to be repaired have been processed, if completed, a 3D model of the local area to be repaired in both PLY and OBJ formats will be generated; otherwise, the next layer will be taken as the current layer, the current position line will be taken as the starting position line of the current layer, and the process will proceed to step c.

8. The method for restoring a three-dimensional model of a city wall ruin according to claim 4, characterized in that: The non-connected deletion mode: deletes m grids from the dense grid at the selected location in one window, and adds the deleted m grids to another window; The connectivity deletion mode is as follows: any two triangular meshes in a dense mesh have two common vertices; in one window, the selected meshes are connected to neighboring meshes to form a local mesh connectivity region, and the mesh connectivity region is deleted; in another window, the deleted mesh connectivity region is added; if the repair software is in connectivity deletion mode and is in the state of saving the dense brick model or the center point cloud of the local area to be repaired, then the deleted mesh region or center point cloud is saved to the file according to the set file name; based on the connectivity of the deleted mesh region, a brick has at least one dense mesh connectivity region, and the dense mesh, its index, and the index of the corresponding original mesh are saved to the corresponding number of brick files; e. The connection deletion mode: In the above non-connected deletion mode, in one window, n deletion operations are performed. Each time, m grids in the dense grid at the selected location are deleted. The center of the center point of the m grids is calculated. The n centers are used as the n endpoints of the line segment to form (n - 1) line segments. The grids in the cylinder with the above line segments as the central axis and radius r are deleted. The deleted grids are added in another window.

9. The method for restoring a three-dimensional model of a city wall ruin according to claim 4, characterized in that: In step b of step S3, if the non-connection deletion mode is inverted, i.e., from 1 to 0, or from 0 to 1; 0 represents the connection deletion mode, and each time the center point of the deleted grid in the above non-connection deletion mode is saved as the endpoint of the line segment in the connection deletion mode; 1 represents the non-connection deletion mode; if the non-connection deletion mode is inverted, i.e., from 1 to 0, or from 0 to 1; 1 represents the non-connection deletion mode, and 0 represents the connected deletion mode; at the same time, the above connection deletion mode operation is performed as needed; finally, the state of saving the dense grid model of the city bricks or the center point cloud of the local area to be repaired is inverted, i.e., from 1 to 0, or from 0 to 1; 1 represents saving the state of the grid model or center point cloud, and the file name of the saved grid model or center point cloud is set; 0 represents not saving the state of the grid model or center point cloud.

10. The method for restoring a three-dimensional model of a city wall ruin according to claim 5, characterized in that: In step a of step S4, the center point cloud of the local area to be repaired has connectivity, that is, the points of the center point cloud of the repaired area correspond one-to-one with the triangular meshes in the dense mesh model, and the meshes have two common points, that is, they are connected. Then the center points corresponding to the meshes in the center point cloud are connected. On the other hand, the two center points in the center point cloud correspond to two unconnected modules in the OBJ module, but the distance between the two center points is less than the resolution of the center point cloud. Then the two center points are connected.

Images