A method for constructing complex models of earthen archaeological sites based on COLMAP

Abstract

This invention discloses a method for constructing complex models of earthen ruins based on COLMAP. By collecting data and processing video frames, the open-source software COLMAP, using an incremental SFM algorithm, performs correspondence search and incremental reconstruction on the Suoyangcheng earthen ruin photo dataset to obtain point cloud data. This point cloud data is then imported into Geomagic Studio to generate a triangular mesh surface model describing the surface of the earthen ruins. The triangular mesh surface model is simplified to minimize the number of point clouds without affecting its accuracy and features. The model is then divided into regions, and NURBS is used to fit the triangular mesh surfaces within each region. This method not only creates complex earthen ruin models but also effectively solves the incompatibility problem with ANSYS models, reduces the time cost required for CFD preprocessing, and resolves the memory overflow problem in numerical simulation preprocessing caused by the complexity of the Suoyangcheng model.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional image processing technology, and specifically to how to construct a complex model of an earthen site in the preprocessing stage of numerical simulation analysis of local wind field at a large earthen site in the field. It relates to a method for constructing a complex model of an earthen site based on COLMAP. Background Technology

[0002] Earthen sites are a type of cultural heritage with extremely high research value. The main structure of large-scale earthen sites in the field is usually constructed of rammed earth. Earthen site artifacts scattered along the Silk Road in Northwest my country have been abandoned for a long time due to their antiquity, resulting in severe structural damage, including erosion by wind and sand, burial by shifting sands, and coverage by vegetation. Among them, Suoyang City, with its grand scale, complex structure, and relatively good preservation, has received high attention from the cultural relics and archaeology community and has been listed as a World Heritage Site, gaining worldwide fame. The area where Suoyang City is located experiences strong winds and frequent natural disasters, along with other human factors, causing significant damage to the walls, leading to weathering, erosion, and collapse. Therefore, the complexity of the surface texture and the diversity of the structure of earthen sites make it difficult to quickly and accurately construct three-dimensional models of them.

[0003] Due to limitations imposed by the management system for earthen archaeological sites, the on-site environment, and the complexity and diversity of surface textures and structures, manual measurement and modeling face significant challenges, including extreme difficulty in fieldwork, high workload, low efficiency, and operational inconvenience. While 3D laser scanning can acquire geometric information about the object's surface with high precision, it requires multiple scans of the earthen archaeological site, and the equipment is expensive and bulky. Furthermore, earthen archaeological site models constructed through third-party methods suffer from incompatibility issues with direct import into general-purpose computational fluid dynamics software.

[0004] Currently, models of earthen sites are often built through manual measurement or 3D laser scanning. However, for large earthen sites in the field, traditional manual measurement requires multiple measurements of the entire scene, while 3D laser scanning equipment is expensive and bulky, making it not the best choice. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a method for constructing complex models of earthen sites based on COLMAP. This method can acquire point cloud data of earthen sites relatively quickly and at low cost, and further process the point cloud data using Geomagic to obtain high-quality three-dimensional models of earthen sites.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for constructing complex models of earthen archaeological sites based on COLMAP, characterized by the following steps:

[0008] Step S1: Data collection: Use a drone equipped with a high-resolution camera to collect videos of the walls of the Suoyangcheng earthen site, and select videos that meet the conditions of good overlap, clear images, good lighting and no overexposure.

[0009] Step S2: Frame extraction processing of the video: Convert the video into photo format and create a photo dataset of the Suoyangcheng earthen site;

[0010] Step S3: Based on the open-source software COLMAP with incremental SFM algorithm, perform correspondence search and incremental reconstruction on the Suoyangcheng earthen site photo dataset to obtain the Suoyangcheng earthen site point cloud data.

[0011] Step S4: Import the point cloud data of the Suoyangcheng earthen site into Geomagic Studio to generate a triangular mesh model that can describe the surface of the earthen site; the specific implementation steps include:

[0012] Step S41: Delete irrelevant point cloud data outside the study area;

[0013] Step S42: Repair the point cloud data of the study area to remove model defects such as nails, holes, and intersecting triangles on the triangular mesh patches;

[0014] Step S5: Simplify the triangular mesh model by minimizing the number of point clouds without affecting its accuracy and features. Specific implementation steps include:

[0015] Step S51: Reduce the point cloud data of the study area using curvature sampling;

[0016] Step S52: Reduce the number of point clouds to the minimum number that can preserve the structural details of the Suoyang City earthen site by reducing the reduction scale by 50% each time.

[0017] Step S6: Divide the model into regions, and then perform surface fitting on the triangular mesh patches within each region using NURBS; the specific implementation steps include:

[0018] Step S61: Divide the model into several large regions based on the curvature changes of the model;

[0019] Step S62: Divide the large region into the final small regions, and use NURBS to perform surface fitting on the mesh patches of the small regions.

[0020] According to the present invention, step S1 is specifically implemented by the following steps:

[0021] Step S11: The drone aerial photography method involves determining a flight route based on the research area of ​​Suoyang City's walls, and taking comprehensive aerial photos of Suoyang City from high altitude without any contact.

[0022] Step S12: After the drone captures video footage, all aerial photos are examined, and images that are relatively complete around the wall are selected as the original video data of the wall.

[0023] Specifically, step S2 includes the following steps:

[0024] Step S21: Use the software Potplayer to process the images captured by the drone in 3-second time units to obtain multiple JPG format photos;

[0025] Step S22: Name the captured photos according to the time of file capture, and use the prefix "image" for all photos.

[0026] The specific implementation steps of step S3 include:

[0027] Step S31: Import the earthen site image dataset into COLMAP and select the SIMPLE_RADIAL camera model for feature point extraction;

[0028] Step S32: Use exhaustive matching to achieve fast and efficient feature point matching and obtain point cloud data.

[0029] This invention presents a method for constructing complex models of earthen ruins based on COLMAP, effectively solving the incompatibility issue when using third-party 3D model data in ANSYS applications. While ensuring modeling accuracy, it is designed specifically for the unique environment of large-scale earthen ruins in the field, exploring more efficient and easier-to-operate modeling methods, and resolving the memory overflow problem in preprocessing for numerical simulation caused by the complexity of the Suoyang City model. The resulting technological innovation lies in:

[0030] (1) An efficient and convenient method for obtaining point cloud data of earthen sites is proposed, which is applicable to the complex environment of large earthen sites in the field.

[0031] (2) Repair and simplify the point cloud data to improve the quality of the triangular mesh and the speed of computer processing.

[0032] (3) By fitting the triangular mesh surface of the earthen site with NURBS, the problem of not being able to import it into general computational fluid dynamics software is effectively solved, which facilitates the subsequent structural wind load on the earthen site and provides scientific theoretical support for the preventive protection of the earthen site. Attached Figure Description

[0033] Figure 1This is a flowchart of the method for constructing complex models of earthen ruins based on UAVs and COLMAP according to the present invention;

[0034] Figure 2 These are some images from the Suoyangcheng earthen site dataset;

[0035] Figure 3 These are partial images of the point cloud data of the Suoyang city wall;

[0036] Figure 4 These are before-and-after images of the nail-shaped object being processed. The left image is before processing, and the right image is after processing.

[0037] Figure 5 These are before-and-after images showing the hole treatment process. The left image is before treatment, and the right image is after treatment.

[0038] Figure 6 This is an image processed by curvature sampling to reduce the point cloud data of the study area.

[0039] Figure 7 These are images of an earthen site model fitted using NURBS surfaces;

[0040] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. Detailed Implementation

[0041] The applicant's research found that the stereoscopic vision technology COLMAP can be used for rapid measurements with lightweight drones, making it more suitable for large-scale outdoor scenes like Suoyang City.

[0042] See Figure 1 This embodiment presents a method for constructing a complex model of an earthen archaeological site based on COLMAP, which specifically includes the following steps:

[0043] Data collection:

[0044] In the field of cultural relic preservation, laser scanning and multi-view methods are commonly used for three-dimensional reconstruction of earthen sites. However, due to the high cost and large size of laser scanning equipment, it is not suitable for the research area of ​​the Suoyang city wall. Therefore, this embodiment adopts multi-view methods. Figure 3 The 3D reconstruction technology was used to reconstruct the Suoyang city wall, which not only meets the accuracy requirements of subsequent CFD simulation calculations, but also reduces the project budget and the modeling difficulty for researchers.

[0045] In the research area of ​​the eastern inner wall, the earthen ruins walls can reach heights of 5-6 meters, but original wall data is lacking. Handheld cameras cannot capture data of the top of the walls, leading to incomplete modeling data. Furthermore, climbing to these heights poses safety risks to researchers. To address this challenge, this embodiment employs drone aerial photography, overcoming the limitations of handheld devices while reducing safety risks. This allows for safe, efficient, and comprehensive data collection of the Suoyang City wall, improving the completeness of the modeling data.

[0046] With the continuous development of computer communication technology and various new sensors, drone photography technology is also constantly improving. Using drones allows for the rapid and flexible acquisition of high-resolution images without the need for physical contact with the subject, making it extremely convenient. In this embodiment, a civilian drone manufactured by DJI was selected. It is equipped with a 12.4-megapixel camera and a 30-minute flight endurance. Furthermore, it features automatic obstacle avoidance, enabling it to capture and record high-altitude scenes even at high altitudes or in locations with poor visibility, providing significant convenience for researchers.

[0047] According to the current "Specifications for Low-Altitude Digital Aerial Photography" CH / T 3005-2021, the following specifications must be followed within the research area of ​​the Suoyangcheng earthen site: The drone should be controlled at a height of approximately 5 meters above the highest point of the wall to avoid dust or other impurities interfering with the flight and to ensure flight safety. To obtain complete wall data, the drone should be laid out along the long side of the research area using a parallel flight path. Furthermore, to avoid the loss of details of the earthen site walls due to strong ground reflections, photography should be conducted 1 to 2 hours before and after noon. Overlap is one of the key factors in ensuring the quality of aerial photographs, referring to the degree of overlap between adjacent images along the flight direction. Generally, the main overlap should be between 60% and 90%, with a minimum of 53%; the lateral overlap should be between 20% and 60%, with a minimum of 8%. Insufficient overlap should be avoided to prevent the need for repeated flights, thus delaying the progress of scientific research.

[0048] After the drone captures video footage, all aerial images are reviewed, and images that relatively completely circle the wall are selected as the original video data. According to "Structure-From-Motion Revisited," the required images should have good texture, similar lighting conditions, high visual overlap, varied angles, and distinctive features; the image format should be JPG. Using Potplayer software, clicking "Video"—"Image Capture"—"Continuous Capture" extracts multiple JPG images from the drone-captured footage in 3-second intervals. (Some images are from the Suoyangcheng database, such as...) Figure 2The method for naming files is based on the file capture time, with the prefix "image".

[0049] Acquisition of point cloud data:

[0050] According to the official COLMAP 3.8 documentation tutorial, first open the COLMAP visualization interface and create a database. Select the corresponding path of the captured photo from the dataset collection from the Images section. Next, extract feature points. COLMAP supports various camera models with varying complexities. If the camera parameters of the selected photos are unknown, the SIMPLE_RADIAL model can be used. This model can simulate different camera parameters for each image, achieving more accurate image correction. If each photo has camera parameters, the OPENCV model can be used, and the camera parameters can be estimated through COLMAP. If a fisheye camera is used, the THIN_PRISM_FISHEYE model should be selected. For image datasets extracted from video, the SIMPLE_RADIAL model is the most suitable choice because the camera parameters are unknown.

[0051] After feature point extraction, feature matching is required, a step that typically consumes a significant amount of time. To adapt to different input scenarios, COLMAP provides multiple matching modes. For example, when the dataset is small, exhaustive matching is the most suitable choice, as it can quickly and effectively achieve good matching results. Specifically, with a maximum of only a few hundred images, each image is compared with all other images to find the best match. This method is both effective and efficient. Furthermore, if the collected photo dataset is continuous, meaning each frame has sufficient visual information, then unlike exhaustive matching, it is not necessary to match each image with all other images; sequential matching is sufficient. Before performing sequential matching, a trie needs to be trained, which can significantly shorten the image matching time. In this embodiment, there are 435 images, making the first matching method the most suitable. It primarily matches based on image features, allowing for more efficient image matching and reducing repetitive data entry. The final point cloud data of the Suoyang City wall is shown in the figure.

[0052] Point cloud data repair

[0053] Importing these point cloud data into Geomagic Studio and encapsulating them generates a triangular mesh model describing the surface of the artifact. However, environmental occlusions can introduce defects, such as nail-like structures near bushes that disrupt the smoothness of the mesh, and intersecting triangles and other mesh flaws in areas with significant point cloud fluctuations. Furthermore, denoising the point cloud data can result in numerous holes on the mesh surface. To address these issues, the triangular mesh model needs to be smoothed, repaired, and intersecting triangles eliminated to improve mesh quality and stability and prevent surface fitting failure. Smoothing algorithms adjust the vertex positions of the mesh to create a smoother surface; mesh repair algorithms fix defects in the model; and triangle reconstruction algorithms generate new triangles from the point cloud data, filling holes and eliminating intersecting triangles. These algorithms can be used in combination to achieve optimal repair results.

[0054] (1) Handling nails

[0055] Geomagic Studio software offers a function to remove sharp points from triangular mesh patches for data repair. This function transforms sharp points into smooth surfaces, restoring the data to a more complete state. In addition, the software provides a lasso tool for manually selecting and deleting sharp points and filling any resulting holes, thus achieving data repair. Figure 4 .

[0056] (2) Eliminate intersecting triangles

[0057] To minimize data loss after removing the intersecting triangular regions and to maintain the smoothness and curvature variation of the surrounding surface, a selection tool was used to precisely select these regions, and a hole-filling method was employed to fill the holes left behind by the removal of the intersecting triangular regions. This ensures the integrity and smoothness of the surface while minimizing the impact on the data.

[0058] (3) Treating holes

[0059] To improve the completeness and targeting of the filling process, first perform a "full filling" to fill less complex holes. For example... Figure 5 As shown, the complex holes caused by honeycomb erosion and collapse of the wall are then manually filled one by one. According to the different characteristics of the complex holes, the filling methods are divided into the following three types:

[0060] 1) Internal holes: Select the edge segment that needs to be filled, and you can completely fill it;

[0061] 2) Boundary holes: Define one point on the edge of the hole as the starting point for filling and select another point on the edge as the ending point for filling to determine the filling boundary area;

[0062] 3) Tower Bridge: Select a point on the edge and drag a bridge to the other side of the edge. The hole can be artificially divided into several holes that can be filled individually according to the curvature characteristics of the surrounding area, and then accurately filled.

[0063] In summary, smoothing, repairing, and eliminating intersecting triangles in triangular mesh models can effectively improve the quality and stability of the mesh model, avoid failure in the crucial step of surface fitting, and thus ensure the accuracy and reliability of subsequent data analysis.

[0064] Simplified processing of point cloud data:

[0065] After processing the image database using COLMAP, a large amount of point cloud data is obtained, containing structural details of the Suoyangcheng earthen site. However, the sheer size of this data slows down computer processing; for example, creating the computational domain in ANSYS requires significant resources, and subsequent mesh generation becomes more difficult. To address this issue, point cloud thinning techniques can be used to remove some point cloud data, thereby reducing the data volume. However, it is crucial to ensure that the thinned point cloud data retains the structural details of the Suoyangcheng earthen site to avoid adverse effects on subsequent calculations and modeling.

[0066] To reduce the number of point cloud data and improve model processing efficiency, three sampling methods can be used: uniform sampling, curvature sampling, and random sampling. These sampling methods can minimize the amount of point cloud data without affecting the accuracy and features of the triangular mesh model. Uniform sampling is a uniform sampling method that can effectively reduce the number of point cloud data, but may affect the local features of the model; curvature sampling can analyze the curvature of the model to determine areas that require denser sampling and areas that can be sampled less; random sampling can quickly reduce the number of point cloud data without affecting model quality. In cultural relic modeling, curvature sampling is often used to reduce the point cloud data of the study area, and the reduction ratio can be determined according to a certain scale. Taking the Suoyang city wall as an example, the number of meshes before reduction is 484231. Figure 6 As shown, after several reductions, the number decreased to approximately 50,000. Choosing a suitable sampling method is crucial for 3D digital reconstruction and subsequent model processing, as it can improve computational efficiency and ensure model quality.

[0067] Surface reconstruction of point cloud data:

[0068] Surface reconstruction is the core step in the reverse reconstruction of a 3D model. The surface features of earthen ruins are highly complex; therefore, using a single low-order surface cannot accurately describe its surface structure, while high-order surfaces are prone to deformation. Therefore, multiple low-order surfaces are combined to ensure surface connectivity and effectively fit the mesh patches. When dividing the mesh patches, appropriate division based on size is necessary. Too few surface patches cannot fit the mesh patches, while too many surface patches increase the computational burden on the computer, hindering the division of the fluid domain for downstream applications. Therefore, the curvature change of each divided region should be as uniform as possible to ensure surface smoothness and accuracy. Specifically, the model can be divided into several large regions as required, and then the final surface patches can be sequentially divided within these large regions. The size of the divided surface patches should be appropriate, fitting the mesh patches while minimizing data and computational load for subsequent modeling and calculation. The earthen ruin model fitted using NURBS surfaces is shown below. Figure 7 As shown.

[0069] in conclusion:

[0070] In summary, the method for constructing complex models of earthen archaeological sites based on COLMAP presented in this embodiment utilizes the application of 3D digital reconstruction technology in cultural relic preservation. High-resolution imagery is captured by drones, point cloud data is obtained through COLMAP processing, and triangular mesh patches are generated, repaired, and simplified using Geomagic. Finally, a high-resolution 3D model is obtained through NURBS surface fitting. This model can be directly used for ANSYS numerical analysis, thereby reducing the workload in the CFD preprocessing and modeling stage.

[0071] To ensure the accuracy and quality of the reconstructed earthen site model during data processing and surface reconstruction, an optimization method specifically for earthen sites is proposed. This method is not only applicable to the Suoyangcheng earthen site but can also support the establishment of 3D digital models for other large-scale earthen sites in the field. Multi-view technology has great potential in the application of earthen site artifacts. By integrating multiple views of earthen site artifacts, a more complete and accurate 3D model can be constructed, providing new ideas and methods for establishing 3D models in wind field analysis of earthen sites. This method can not only help researchers better understand the structure and characteristics of earthen site artifacts but also provide a reliable model for ANSYS numerical analysis.

Claims

1. A method for constructing complex models of earthen archaeological sites based on COLMAP, characterized in that, Includes the following steps: Step S1: Data collection: Use a drone equipped with a high-resolution camera to collect videos of the earthen ruins walls, and select videos that meet the conditions of good overlap, clear images, good lighting and no overexposure. The specific implementation steps of step S1 include: Step S11: The drone aerial photography method involves determining a flight route based on the research area of ​​the earthen site walls, and taking comprehensive aerial photos of the earthen site without any contact. Step S12: After the drone captures video footage, all aerial photos are examined, and images that are relatively complete around the wall are selected as the original video data of the wall. Step S2: Frame extraction processing of the video: Convert the video into photo format to create a dataset of photos of the earthen ruins; the specific implementation steps of step S2 include: Step S21: Use the software Potplayer to process the images captured by the drone in 3-second time units to obtain multiple JPG format photos; Step S22: Name the captured photos according to the time of file capture, with the prefix "image"; Step S3: Using the open-source software COLMAP based on the incremental SFM algorithm, perform correspondence search and incremental reconstruction on the earthen site photo dataset to obtain earthen site point cloud data; the specific implementation steps of step S3 include: Step S31: Import the earthen site image dataset into COLMAP and select the SIMPLE_RADIAL camera model for feature point extraction; Step S32: Use exhaustive matching to achieve fast and efficient feature point matching and obtain point cloud data; Step S4: Import the point cloud data of the earthen site into Geomagic Studio to generate a triangular mesh model that describes the surface of the earthen site; the specific implementation steps include: Step S41: Delete irrelevant point cloud data outside the study area; Step S42: Repair the point cloud data of the study area, removing nail-like objects, holes, and defects in the intersecting triangle model on the triangular mesh surface; Step S5: Simplify the triangular mesh model by minimizing the number of point clouds without affecting its accuracy and features. Specific implementation steps include: Step S51: Reduce the point cloud data of the study area using curvature sampling; Step S52: Reduce the number of point clouds to the minimum number that can preserve the structural details of the earthen site, using a reduction scale of 50% for each step. Step S6: Divide the model into regions, and then perform surface fitting on the triangular mesh patches within each region using NURBS; the specific implementation steps include: Step S61: Divide the model into several large regions based on the curvature changes of the model; Step S62: Divide the large region into the final small regions, and use NURBS to perform surface fitting on the mesh patches of the small regions.

Images