Irregular surface soil Great Wall remaining volume measurement method and device, electronic equipment and computer storage medium

Abstract

The embodiment of the invention provides an irregular surface soil Great Wall remaining volume measurement method and device, electronic equipment and a computer storage medium. The method comprises the following steps: acquiring laser radar point cloud data and DOM data of the Great Wall ruins through an unmanned aerial vehicle; based on the DOM data, extracting vegetation space range data on and around the wall body of the Great Wall; and generating laser radar point cloud data of the vegetation-free Great Wall ruins and the surrounding environment, obtaining ruins body laser radar point cloud data from the laser radar point cloud data, and calculating the volume of the Great Wall ruins. According to the method, the three-dimensional model and the volume of the great wall remaining in a large range can be effectively obtained, the surface morphological change of the great wall in different periods can be compared and monitored, and the refined development of great wall protection research work can be promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of irregular surface measurement, and in particular, to a method, device, electronic device and computer storage medium for measuring the volume of remains of earth Great Wall with irregular surfaces. Background Technique

[0002] The Great Wall is the largest and most widely distributed linear cultural heritage in existence, distributed in 404 counties of 15 provinces (autonomous regions, municipalities directly under the Central Government). In the investigation, research and protection work, the quantitative description of the existing situation of the Great Wall is often involved, such as the measurement of the volume of remains. Due to the long age and severe weathering of the existing Great Wall walls, there are large differences in the height, width, etc. of the walls, the surfaces are very irregular, and the terrain along the line is complex and there are many shrubs and trees, resulting in the accurate measurement of the volume becoming a technical difficulty. The northwest earth Great Wall site is the most typical. There is an urgent need to design a method that can accurately measure the volume of the remains of the northwest earth Great Wall with irregular surfaces in order to better study and protect the Great Wall site.

[0003] Since 2006, the National Cultural Heritage Administration has carried out the surveying and mapping and resource investigation of the Ming Great Wall, but has not involved the relevant three-dimensional shape modeling work of the site itself. In recent years, the Key Laboratory of the Ministry of Culture and Tourism has carried out the construction of a comprehensive three-dimensional model database of the entire line environment of the Ming Great Wall, but has not further carried out the fine extraction work of the Great Wall walls. When some scholars carry out the surveying and mapping research on important sections of the Great Wall, the measurement of the volume of the remains of the Great Wall is generally limited to the rough measurement and estimation at the contour level, selecting areas with relatively simple environments and no excessive vegetation interference, or selecting sections of the Great Wall with relatively regular shapes and easy to measure. With the rapid development of unmanned aerial vehicles (UAVs) and remote sensing sensors, there is technical support for the fine surveying, mapping, protection and digitalization of the Great Wall. However, the current applications are only limited to small-scale experiments, and a technical system for general or large-scale surveying and modeling has not been formed yet. For example, important branch technologies such as accurate modeling by removing vegetation interference from the remaining walls and volume surveying and mapping are not yet mature and need to be studied and supplemented. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, device, electronic device and computer storage medium for measuring the volume of remains of earth Great Wall with irregular surfaces, which can effectively obtain the three-dimensional model and volume of the remains of the Great Wall in a large range.

[0005] To achieve the above purpose, the technical solutions adopted in the embodiments of the present invention are as follows:

[0006] In the first aspect, the present invention provides a method for measuring the volume of remains of earth Great Wall with irregular surfaces, the method comprising:

[0007] S1: Obtain the UAV multispectral data and lidar data of the Great Wall site, and preprocess the UAV multispectral data and lidar data to obtain lidar point cloud data and DOM data;

[0008] S2: Based on the DOM data obtained in S1, combined with the deep learning algorithm for image recognition, extract the vegetation spatial range data on and around the Great Wall wall body;

[0009] S3: According to the vegetation spatial range data extracted in S2, remove the vegetation part of the lidar point cloud data to generate lidar point cloud data of the Great Wall ruins and its surrounding environment without vegetation;

[0010] S4: Based on the lidar point cloud data of the Great Wall ruins and its surrounding environment without vegetation generated in S3, perform the extraction of the Great Wall ruins body through the cloth simulation filtering algorithm to obtain the lidar point cloud data of the ruins body;

[0011] S5: Based on the lidar point cloud data of the ruins body obtained in S4, calculate the volume of the Great Wall ruins.

[0012] In an optional implementation manner, S1 includes:

[0013] S11: Conduct an environmental survey to formulate a flight operation plan;

[0014] S12: Design the flight route of the unmanned aerial vehicle;

[0015] S13: Set up an RTK base station;

[0016] S14: Flight verification, including conducting test flights at the edge and center of the survey area to check the quality of POS data and the RTK fixation rate;

[0017] S15: Formal operation, including flying in blocks or continuously according to the coverage range of the RTK base station, conducting unmanned aerial vehicle mapping of the survey area one by one, monitoring the communication status in real time, and obtaining the multi-spectral data and lidar data of the Great Wall ruins by the unmanned aerial vehicle.

[0018] In an optional implementation manner, S2 includes:

[0019] S21: Make label images, including selecting hundreds or thousands of image sample data with the same size according to the image features of vegetation in the DOM data, and extracting the label images of each sample data through interpretation;

[0020] S22: Train the model, including training the deep learning model by randomly selecting 2 / 3 of the made label images through the Swin Transformer deep learning model;

[0021] S23: Verify the model accuracy, including verifying the model accuracy through the remaining label images after the model training is completed.

[0022] In an optional implementation manner, S3 includes:

[0023] S31: Preprocess the lidar point cloud data to obtain the point cloud area with vegetation.

[0024] S32: Select initial seed points in the point cloud area with vegetation to construct an initial ground model.

[0025] S33: Identify the remaining ground points in the point cloud area with vegetation and add them to the initial ground model.

[0026] S34: Verify the accuracy of the initial ground model and generate the lidar point cloud data of the Great Wall site and its surrounding environment without vegetation.

[0027] In an alternative embodiment, S32 includes:

[0028] Divide the point cloud area with vegetation into grids and select the lowest point of each grid as the initial seed point;

[0029] Use the initial seed points to construct an initial ground model.

[0030] In an alternative embodiment, S33 includes:

[0031] Sort the remaining points in the point cloud area with vegetation in ascending order of elevation and judge whether each point is added to the initial ground model one by one. Among them, the judgment condition: if the point meets the following conditions, mark the point as a ground point and add it to the initial ground model.

[0032] d i ≤h max and θ i ≤θ max

[0033] Where:

[0034] d i is the vertical distance from node P i to the nearest initial ground;

[0035] h max is the maximum height difference threshold;

[0036] θ i is the angle between node P i and the normal vector of the initial ground;

[0037] θ max is the maximum angle threshold.

[0038] In an alternative embodiment, S4 includes:

[0039] S41: Initialize the lidar point cloud data of the Great Wall site and its surrounding environment without vegetation to form a cloth.

[0040] S42: Calculate the gravitational direction of the nodes in the fabric;

[0041] S43: Calculate the gravity of the nodes in the fabric based on the gravitational direction;

[0042] S44: Calculate the spring force of the nodes in the fabric;

[0043] S45: Calculate the total force of the nodes in the fabric based on gravity and spring force;

[0044] S46: Update the positions of the nodes in the fabric based on the total force;

[0045] S47: Perform collision detection to obtain the latest ground height of the nodes;

[0046] S48: Repeat S44 to S47 until the fabric reaches a stable state or a preset number of iterations;

[0047] S49: Identify the main body of the Great Wall site based on the updated positions of the nodes and the ground height;

[0048] In a second aspect, the present invention provides a device for measuring the volume of an irregular surface earthen Great Wall relic, the device comprising:

[0049] A data acquisition module, configured to acquire the multi-spectral data and lidar data of the Great Wall site by an unmanned aerial vehicle, and preprocess the multi-spectral data and lidar data of the unmanned aerial vehicle to obtain lidar point cloud data and DOM data;

[0050] A data extraction module, configured to extract the vegetation spatial range data on and around the Great Wall wall based on the DOM data acquired by the data acquisition module and in combination with an image recognition deep learning algorithm;

[0051] A data elimination module, configured to remove the vegetation part point cloud in the lidar point cloud data according to the vegetation spatial range data extracted by the data extraction module, and generate lidar point cloud data of the Great Wall site and its surrounding environment without vegetation;

[0052] A main body extraction module, configured to perform main body extraction of the Great Wall site based on the lidar point cloud data of the Great Wall site and its surrounding environment without vegetation generated by the data elimination module through a fabric simulation filtering algorithm, and obtain the lidar point cloud data of the site main body;

[0053] A volume calculation module, configured to calculate the volume of the Great Wall site based on the lidar point cloud data of the site main body obtained by the main body extraction module.

[0054] In a third aspect, the present invention provides an electronic device, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the method according to any one of the foregoing embodiments when executing the computer program.

[0055] In a fourth aspect, the present invention provides a computer storage medium storing a computer program, which when executed by a processor implements the steps of the method according to any one of the foregoing embodiments.

[0056] The method for measuring the volume of the irregular-surface earth Great Wall remains provided by the embodiments of the present invention belongs to an urgently needed technical method in the current digital protection work of the Great Wall sites, can effectively obtain the three-dimensional models and volumes of the Great Wall remains in a large area, and realizes the comparative monitoring of the surface morphology changes of the Great Wall in different periods, which can promote the refined development of the Great Wall protection research work.

[0057] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0059] Figure 1 Shows a flowchart of the method for measuring the volume of the irregular-surface earth Great Wall remains provided by the embodiments of the present invention;

[0060] Figure 2 Shows a schematic diagram of the composition of the device for measuring the volume of the irregular-surface earth Great Wall remains provided by the embodiments of the present invention;

[0061] Figure 3 Shows a schematic diagram of the composition of the electronic device provided by the embodiments of the present invention.

[0062] Reference numerals: 1 - data acquisition module; 2 - data extraction module; 3 - data elimination module; 4 - body extraction module; 5 - volume calculation module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] To make the objects, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0064] The following will be combined with Figures 1 to 3Describe the method and device for measuring the volume of the remains of the earthen Great Wall with an irregular surface provided in this embodiment.

[0065] As Figure 1 shown, this embodiment provides a method for measuring the volume of the remains of the earthen Great Wall with an irregular surface (hereinafter referred to as: method). Based on the three-dimensional modeling of unmanned aerial vehicle (UAV) lidar and multi-spectral imaging technology, through the data of multi-type sensors of UAV, three-dimensional digital modeling of the earthen Great Wall site with an irregular surface is carried out, and the trees and shrubs on and around the Great Wall wall in the model are removed. Fine extraction and volume mapping of the body of the irregular Great Wall site are carried out, which is applicable to the three-dimensional fine modeling census of the wall of the remains of the irregular Great Wall and the detailed investigation of local sections, as well as the monitoring of pre- and post-period changes through the obtained high-precision three-dimensional model data of the Great Wall wall, which is convenient for popularization and operation. The method specifically includes the following steps:

[0066] S1: Obtain the multi-spectral data and lidar data of the Great Wall site, and preprocess the multi-spectral data and lidar data of the UAV to obtain lidar point cloud data and DOM (full name: Digital OrthophotoMap, Chinese name: Digital Orthophoto Map) data.

[0067] Among them, the lidar point cloud data and DOM data are obtained by using the UAV aerial photography technology, and the specific process is as follows:

[0068] S11: Conduct an environmental survey to formulate a flight operation plan.

[0069] Before the flight operation, relevant topographic maps, images and other materials or data related to the shooting area should be fully collected, the topographic features, climatic conditions, airports, important facilities, etc. of the shooting area should be understood, and on-site operation surveys should be carried out in the operation area. Through comprehensive analysis and research, the airspace conditions of the flight area and the adaptability of the equipment to the task are determined, and a detailed flight operation plan is formulated.

[0070] S12: Design the flight route of the UAV.

[0071] According to the position and scope of the monitoring unit, as well as the flight quality requirements, the flight route is reasonably standardized. The resolution of the obtained DOM data is better than 5 cm, and the lidar point cloud density is better than 500 pts / m 2 , and the width of the image strip is not less than 50 meters on both sides of the Great Wall (referring to the principle that the protection scope is not less than 50 meters in the "Overall Plan for the Protection of the Great Wall" issued by the Ministry of Culture and Tourism and the National Cultural Heritage Administration). In areas with large terrain undulations, the flight routes should be set according to the altitude levels (or adopt the terrain-following flight strategy) to ensure the consistency of the true spatial resolution of the data under different undulating terrain conditions.

[0072] a. Obtain DOM data.

[0073] Calculate and determine the aerial photography altitude of the UAV according to the job safety requirements, data quality, and sensor parameters. The calculation formula is as follows:

[0074] H = (f × GSD) / a

[0075] Where:

[0076] H is the aerial photography altitude, with the unit of m;

[0077] f is the focal length of the sensor lens, with the unit of mm;

[0078] GSD is the imaging resolution, with the unit of m;

[0079] a is the sensor pixel size or point cloud spacing, with the unit of mm.

[0080] b. Obtain LiDAR point cloud data.

[0081]

[0082] h: UAV flight altitude (meters);

[0083] υ: UAV flight speed (meters per second);

[0084] ρ: Point cloud density (points per square meter);

[0085] f: Pulse frequency of the LiDAR (Hz, points per second), the number of laser points emitted by the LiDAR per second, usually ranging from several hundred thousand to several million points per second;

[0086] μ: Scanning efficiency, the proportion of actual effective scanning points, usually ranging from 0.5 to 1;

[0087] θ: Scanning field of view (FOV) of the LiDAR, usually ranging from 30° to 360°.

[0088] S13: Set up an RTK (Full name: Real-Time Kinematic, Chinese name: Real-Time Dynamic) base station.

[0089] Before the UAV flight operation, in order to improve the surveying and mapping accuracy, it is necessary to set up a mobile RTK base station in the surveying and mapping area. The setup process is as follows:

[0090] Preliminary planning: Divide the survey area according to the spacing of 3 to 5 kilometers through satellite maps. Based on DEM (Full name: Digital Elevation Model, Chinese name: Digital Elevation Model) data and basic road data, plan the feasible layout positions of RTK base stations. It is required that there is no terrain obstruction in the signal link between the UAV and the RTK base station during the survey process. Plan the position of one RTK base station for each survey area, and conduct concrete pouring for the control points at the positions. Obtain high-precision coordinates (millimeter level) through static observation (≥2 hours).

[0091] Installation of RTK base station: Install the RTK base station according to the plan, and complete coordinate calculation and signal testing.

[0092] S14: Flight verification: Conduct short-time test flights at the edge and center of the survey area to check the POS (Full name: position and orientation system, Chinese name: positioning and orientation system) data quality and RTK fixation rate.

[0093] S15: Formal operation: Fly in blocks or continuously according to the coverage range of the RTK base station, conduct UAV survey on the survey area one by one, monitor the communication status in real time, and obtain the UAV multispectral data and lidar data of the Great Wall site.

[0094] Among them, the steps for preprocessing the UAV multispectral data and lidar data include:

[0095] Generate DOM data, which specifically includes processes such as denoising, contrast enhancement, equalization of light and color, geometric correction, radiometric correction, and image mosaicking.

[0096] Generate lidar point cloud data, which specifically includes processes such as data parsing, point cloud noise removal, coordinate transformation and format conversion, point cloud segmentation and classification, motion distortion compensation, data compression and storage.

[0097] In this way, when obtaining the lidar point cloud data, high-precision positioning RTK base stations are planned and laid out, providing a solid guarantee for the survey accuracy of the obtained point cloud data, realizing the monitoring of the three-dimensional shape changes of the Great Wall site at the same location at different times based on the lidar point cloud data, and supporting the refined monitoring, evaluation and protection work of the Great Wall site.

[0098] S2: Based on the DOM data obtained in S1, combined with the deep learning algorithm of image recognition, extract the spatial range data of the vegetation (such as trees and shrubs) on and around the Great Wall wall.

[0099] Specifically, digital elevation terrain data without vegetation can be obtained by using lidar point cloud data through a vegetation filtering algorithm. However, the basic principle of removing vegetation from lidar point cloud data is to judge vegetation points based on the elevation mutation (local discontinuity) between adjacent lidar points. Under ordinary natural terrain conditions, vegetation can be well identified. However, when the terrain is complex or there are many interference factors, the discrimination error will be relatively large. For example, in this application, the Great Wall wall, as an artificial building, has obvious elevation mutations, and the discriminated vegetation is close to or on the Great Wall wall. In this case, the discrimination error of the vegetation filtering algorithm will increase significantly. Therefore, in this application, an image recognition deep learning algorithm is used to extract vegetation spatial range data, and the process is as follows:

[0100] S21: Make a label image.

[0101] According to the image features of vegetation in DOM data, hundreds or thousands of image sample data of the same size (usually 512×512 pixels) are selected, and the label images of each sample data are extracted through interpretation.

[0102] S22: Train the model.

[0103] Through the Swin Transformer deep learning model, randomly select 2 / 3 of the made label images for deep learning model training.

[0104] Among them, the Swin Transformer deep learning model is a vision Transformer model proposed by Microsoft Research Asia. It is designed for efficiently processing high-resolution images. Its core innovation lies in the hierarchical structure (English name: Hierarchical Architecture) and shifted window self-attention, which solves the bottleneck of the traditional Transformer model in computational complexity and makes it perform excellently in tasks such as image classification, object detection, and semantic segmentation, especially suitable for fine extraction of vegetation in UAV remote sensing.

[0105] The specific introduction of the core innovation of the Swin Transformer deep learning model is as follows:

[0106] 1) Hierarchical structure

[0107] Limitations of the traditional Transformer model: ViT (English name: Vision Transformer) divides an image into blocks of a fixed size (such as 16×16), but lacks the ability to express multi-scale features.

[0108] Improvements to the Swin Transformer deep learning model: By merging image blocks layer by layer (English name: PatchMerging), a pyramid structure similar to CNN (English name: Convolutional Neural Networks, Chinese name: Convolutional Neural Network) is constructed to form four stages (English name: Stage):

[0109] Stage 1: The input image is divided into 4×4 blocks, and the feature map size is:

[0110]

[0111] Stage 2-4: Gradually merge adjacent blocks, and the feature map size is reduced to:

[0112]

[0113] Supports multi-scale feature extraction and adapts to visual tasks of different granularities (such as local texture and global distribution of vegetation).

[0114] 2) Shift Window Self-Attention

[0115] Window division: The feature map is divided into non-overlapping local windows (such as 7×7 size), and self-attention is calculated independently in each window.

[0116] Computational complexity optimization: The computational complexity within the window is O(M 2 ×N, M is the window size, N is the total number of image blocks), which is much lower than O(N 2 ).

[0117] Window shift: In the adjacent layer, the window is shifted to the lower right corner by half the window size (such as from (0,0) to (3,3)), breaking the window boundary limit.

[0118] Function: Enhance cross-window contextual interaction and avoid local information islands.

[0119] Mathematical expression: Window self-attention formula:

[0120]

[0121] Among them, B is the relative position code to enhance spatial perception.

[0122] S23: Verify model accuracy.

[0123] After the model training is completed, the model accuracy is verified through the remaining labeled images. The verification indicators include accuracy, precision, recall, F1 score, intersection-over-union ratio, etc.

[0124] a. Accuracy: It represents the proportion of the number of samples correctly predicted by the model to the total number of samples. It is one of the most intuitive and commonly used evaluation metrics, and the formula is as follows:

[0125]

[0126] Where:

[0127] TP (True Positive): The number of samples correctly predicted as the positive class (vegetation) by the model;

[0128] TN (True Negative): The number of samples correctly predicted as the negative class (non-vegetation) by the model;

[0129] FP (False Positive): The number of samples incorrectly predicted as the positive class;

[0130] FN (False Negative): The number of samples incorrectly predicted as the negative class.

[0131] b. Precision: Also known as the positive predictive value, it represents the proportion of samples actually being positive among the samples predicted as positive, and the formula is as follows:

[0132]

[0133] Where:

[0134] TP: The number of correctly predicted vegetation pixels;

[0135] FP: The number of non-vegetation pixels incorrectly predicted as vegetation.

[0136] c. Recall: Also known as the sensitivity or true positive rate, it represents the proportion of all samples actually being positive that are correctly predicted as positive by the model, and the formula is as follows:

[0137]

[0138] Where:

[0139] TP: The number of correctly predicted vegetation pixels;

[0140] FN: The number of true vegetation pixels not recognized by the model (missed detections).

[0141] d. F1 Score: It is the harmonic mean of precision and recall, attempting to find a balance between these two metrics, and is particularly suitable for dealing with imbalanced datasets. The formula is as follows:

[0142]

[0143] Where:

[0144] Precision: Precision rate;

[0145] Recall: Recall rate.

[0146] e. Intersection over Union (IoU): Measures the overlap between the predicted result and the ground truth label, and is a very important evaluation metric in semantic segmentation. The formula is as follows:

[0147]

[0148] In the formula:

[0149] TP: Number of correctly predicted vegetation pixels;

[0150] FP: Number of non-vegetation pixels wrongly predicted as vegetation;

[0151] FN: Number of true vegetation pixels not recognized.

[0152] S24: Extract vegetation spatial range data.

[0153] Use the trained model to segment the DOM data of the UAV and extract the vegetation spatial range data.

[0154] S3: According to the vegetation spatial range data extracted by S2, remove the vegetation part of the lidar point cloud data to generate the lidar point cloud data of the Great Wall site and its surrounding environment without vegetation.

[0155] Specifically, S3 includes the following steps:

[0156] S31: Preprocess the lidar point cloud data to obtain the point cloud area with vegetation.

[0157] In software versions above ArcGIS Pro 2.6, use the vegetation spatial range data extracted in S2 to crop the lidar point cloud data to obtain the point cloud area with vegetation.

[0158] S32: Select initial seed points in the point cloud area with vegetation and construct an initial ground model.

[0159] Divide the point cloud area with vegetation into grids (the side length of the grid is L, and L is usually taken as 2 - 3 times the average distance between the initial seed points), and select the lowest point of each grid as the initial seed point. Then the generation formula of the initial seed point is as follows:

[0160]

[0161] Where:

[0162] S0 is the set of initial seed points for constructing the initial Triangulated Irregular Network (TIN).

[0163] P i is the point cloud of the vegetation area.

[0164] P v is the subset of the point cloud of the vegetation area.

[0165] Cell m is the m th grid, and the study area is divided into square grids with side length L.

[0166] z i is the elevation value of point P i (unit: meter).

[0167] Min is to take the point with the lowest elevation in the grid cell as the initial seed point.

[0168] In this way, the initial seed points for the initial selection can be generated using S0 to construct the initial ground model.

[0169] S33: Identify the remaining ground points in the point cloud area with vegetation and add them to the initial ground model.

[0170] Specifically, sort the remaining points in the point cloud area with vegetation in ascending order of elevation, and judge whether to add them to the initial ground model point by point. The judgment method for the remaining ground points is as follows:

[0171] 1) Calculate the vertical distance from the point to the initial ground (triangular surface):

[0172]

[0173] Among them:

[0174] d i is the vertical distance from point P i to the nearest initial ground;

[0175] x i 、y i 、z i are the three-dimensional coordinates of point P i ;

[0176] ax + by + cz + d = 0 is the equation of the initial ground, where a, b, c, and d are coefficients.

[0177] 2) Calculate the angle threshold between the point and the normal vector of the triangular surface:

[0178]

[0179] Among them:

[0180] θi is the angle between the normal vector of the point P i and the normal vector of the initial ground;

[0181] n t is the normal vector of the initial ground;

[0182] n i is the local normal vector of the domain point of the point P i .

[0183] 3) Judgment condition: If the following conditions are met, the point P i is marked as a ground point and added to the initial ground model.

[0184] d i ≤h max and θ i ≤θ max

[0185] where:

[0186] h max is the maximum height difference threshold (such as 0.5 - 2.0 m, take a low value in flat areas and a high value in mountainous areas);

[0187] θ max is the maximum angle threshold (such as 5° - 15°).

[0188] S34: Verify the accuracy of the initial ground model and generate lidar point cloud data of the Great Wall site and its surrounding environment without vegetation.

[0189] Specifically, use the ground truth points to calculate the classification accuracy in the initial ground model.

[0190]

[0191] where:

[0192] TP (full name: True Positive) is the number of points correctly classified as ground points;

[0193] TN (full name: True Negative) is the number of points correctly classified as non - ground points (such as vegetation points);

[0194] FP (full name: False Positive) is the number of non - ground points misclassified as ground points;

[0195] FN (full name: False Negative) is the number of ground points misclassified as non - ground points.

[0196] In this way, when the accuracy (precision) meets the standard, it is determined that the initial ground model contains the lidar point cloud data of the Great Wall site without vegetation and its surrounding environment; when the accuracy (precision) does not meet the standard, S3 is re-executed.

[0197] In S2 and S3, in order to avoid the interference of trees and shrubs on the Great Wall wall or around it to the 3D modeling of the Great Wall site, through the deep learning algorithm of image recognition, the spatial range data of trees and shrubs are accurately extracted, and further calculation is carried out to obtain the 3D point cloud data of the Great Wall site and its environment that is not affected by tall vegetation. Compared with directly using the vegetation filtering algorithm, it effectively avoids the problem that it is difficult to distinguish the vegetation and wall point clouds due to the common elevation mutation of the wall and tall vegetation relative to the natural ground surface.

[0198] S4: Based on the lidar point cloud data of the Great Wall site without vegetation and its surrounding environment generated in S3, through the cloth simulation filtering algorithm, the extraction of the main body of the Great Wall site is carried out to obtain the lidar point cloud data of the main body of the site.

[0199] Specifically, S4 includes the following steps:

[0200] S41: Initialize the lidar point cloud data of the Great Wall site without vegetation and its surrounding environment to form cloth.

[0201] Regard the point cloud data as a surface and flip it. Simulate covering a layer of cloth on the flipped surface and initialize the cloth grid. The grid size is 1 to 2 times the average distance of the point cloud, and the position of the initial "cloth" is above the highest point of the initial ground.

[0202] S42: Calculate the gravity direction of the nodes in the cloth.

[0203] Determine a gravity direction vector:

[0204] S43: Calculate the gravity of the nodes in the cloth based on the gravity direction.

[0205] For each node P on the cloth i , calculate the gravity it receives:

[0206]

[0207] Among them, m is the mass of the node.

[0208] S44: Calculate the spring force of the nodes in the cloth.

[0209] For each node P on the cloth i , calculate the spring force between it and the adjacent node P j . The spring force F spring,ij can be calculated by Hooke's law:

[0210]

[0211] Among them, k is the spring coefficient, is the elongation of the spring, that is, the difference between the distance between P i and P j and their initial distance.

[0212] S45: Calculate the total force on the nodes in the fabric based on gravity and spring force.

[0213] For each node P i on the fabric, calculate the total force F total it receives, including gravity and spring force:

[0214]

[0215] In the formula, neighbors(P i ) is the set of adjacent nodes of node P i .

[0216] S46: Update the positions of the nodes in the fabric grid based on the total force.

[0217] According to Newton's second law a i = F total / m, calculate the acceleration a i of each node, and then update the position of the node:

[0218]

[0219] Among them, is the velocity of the node at time t, △t is the time step, is the position of the node at time t.

[0220] S47: Collision detection to obtain the ground height of the nodes.

[0221] For each grid node P i , search for the nearest point cloud qk = (x , y k , z k , z k ) vertically projected below its position. If z k ≥ h i , node P i is "blocked", then record z k as the ground height corresponding to node P i , where h i is the latest vertical height of node P i .

[0222] S48: Repeat S44 to S47 until the fabric reaches a stable state or a preset number of iterations is reached.

[0223] S49: Identify the main body of the Great Wall site based on the updated positions of the nodes and the ground height.

[0224] After the fabric reaches a stable state or a preset number of iterations is reached, identify the points in the fabric where the height of the nodes is higher than or equal to the surface of the surrounding point cloud. These nodes are part of the main body of the Great Wall site.

[0225] The setting of the parameters of the fabric simulation algorithm directly affects the filtering effect. The main custom parameters of this method are the grid resolution of the point cloud data and the spring coefficient: 1) The grid resolution represents the size of the grid for simulating the fabric. The larger this parameter is set, the rougher the simulated terrain will be. On the contrary, the smaller it is, the finer it will be. Therefore, for terrains with different undulation sizes, different grid sizes can be set; 2) The spring coefficient is the fabric hardness. The larger the value, the tighter the fabric. Adjust the coefficient value to adapt to different terrain conditions.

[0226] Through the above calculations, the lidar point cloud data of the main body of the Great Wall site is obtained.

[0227] In S4, a fabric filtering algorithm generally used to extract vegetation is adopted for extracting the information of the main body of the Great Wall, which belongs to the migration and innovation application of technology. By adjusting the parameters, this method can adapt to the extraction of the lidar point cloud information of the main body of the Great Wall under various terrain conditions. The method has good universality and is more suitable for extracting the information of the Great Wall sites under the diverse terrain background in the northwest region.

[0228] S5: Calculate the volume of the Great Wall site based on the lidar point cloud data of the site main body obtained in S4.

[0229] Specifically, S5 includes the following steps:

[0230] S51: Triangulate the lidar point cloud data to convert the discrete point cloud data into a continuous geometric body and construct a three-dimensional surface model composed of multiple triangles.

[0231] S52: Decompose the three-dimensional surface model into small tetrahedrons and calculate the volume of each tetrahedron.

[0232] The three-dimensional surface model is decomposed into small tetrahedrons. By calculating the volume of each tetrahedron and summing them up, the total volume is obtained. The volume of a tetrahedron can be calculated from the coordinates of its four vertices. The formula is:

[0233]

[0234] Where, are vectors from one vertex of a tetrahedron to the other three vertices, · represents the dot product, and x represents the cross product.

[0235] S53: Sum the volumes of all tetrahedrons to obtain the total volume of the entire three-dimensional model:

[0236]

[0237] In the formula: V is the volume of the Great Wall site to be obtained, and v i is the volume of the i-th tetrahedron.

[0238] The volume of the main body of the Great Wall site is obtained by calculating through the above method.

[0239] The beneficial effects of the method for measuring the volume of the irregular surface earth Great Wall remains provided in this embodiment include:

[0240] 1. This method belongs to the urgently needed technical method in the current digital protection work of the Great Wall site. It can effectively obtain the three-dimensional model and volume of the Great Wall remains in a large area, and realizes the comparative monitoring of the surface morphology changes of the Great Wall in different periods, which can promote the refined development of the Great Wall protection research work;

[0241] 2. The basic equipment technologies of the drone multispectral and lidar required by this method are mature, providing a basic technical guarantee for the feasibility of the method;

[0242] 3. The technical process of this method is easy to master, with convenient operation, high accuracy, and strong versatility. It can not only serve the investigation of the earth Great Wall remains in scientific research work, but also provide a technical method for the volume census of the earth Great Wall remains in a large area in the northwest for cultural relics protection.

[0243] Next, the device for measuring the volume of the irregular surface earth Great Wall remains provided in this embodiment will be described. The device for measuring the volume of the irregular surface earth Great Wall remains described below can be mutually corresponding and referred to with the method for measuring the volume of the irregular surface earth Great Wall remains described above.

[0244] As Figure 2 shown, the device for measuring the volume of the irregular surface earth Great Wall remains includes a data acquisition module 1, a data extraction module 2, a data rejection module 3, a main body extraction module 4, and a volume calculation module 5.

[0245] The data acquisition module 1 is used to acquire the drone multispectral data and lidar data of the Great Wall site, and preprocess the drone multispectral data and lidar data to obtain lidar point cloud data and DOM data;

[0246] The data extraction module 2 is used to extract the vegetation spatial range data on and around the Great Wall wall based on the DOM data acquired by the data acquisition module 1 in combination with the deep learning algorithm of image recognition;

[0247] The data elimination module 3 is used to remove the vegetation part of the lidar point cloud data according to the vegetation spatial range data extracted by the data extraction module 2, and generate lidar point cloud data of the Great Wall site and its surrounding environment without vegetation.

[0248] The main body extraction module 4 is used to extract the main body of the Great Wall site through the cloth simulation filtering algorithm based on the lidar point cloud data of the Great Wall site and its surrounding environment without vegetation generated by the data elimination module 3, and obtain the lidar point cloud data of the site main body.

[0249] The volume calculation module 5 is used to calculate the volume of the Great Wall site based on the lidar point cloud data of the site main body obtained by the main body extraction module 4.

[0250] Figure 3 The schematic diagram of the physical structure of an electronic device is exemplified. The electronic device can be a smart terminal, and its internal structure diagram can be as Figure 3 shown. The electronic device includes a processor, an internal memory, and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above-mentioned method for measuring the volume of the irregular surface earth Great Wall remains is realized.

[0251] Those skilled in the art can understand that Figure 3 the structure shown in

[0252] is only a block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the electronic device to which the solution of the present invention is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0253] On the other hand, this embodiment also provides a computer storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned method for measuring the volume of the irregular surface earth Great Wall remains is realized.

[0254] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. This computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above various methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory.

[0255] By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0256] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0257] The above embodiments only represent several implementation manners of the present invention, and the description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

Claims

1. A method for measuring the volume of the Great Wall remains on an irregular surface, characterized in that: The method comprises: S1: Obtain UAV multispectral data and LiDAR data of the Great Wall ruins, and pre-process the UAV multispectral data and LiDAR data to obtain LiDAR point cloud data and DOM data; S2: Based on the DOM data obtained by S1, combined with the image recognition deep learning algorithm, the spatial range data of vegetation on and around the Great Wall is extracted; S3: According to the vegetation spatial range data extracted by S2, the vegetation point cloud in the lidar point cloud data is removed to generate the lidar point cloud data of the Great Wall ruins and the surrounding environment without vegetation; S4: Based on the LiDAR point cloud data of the Great Wall ruins and the surrounding environment without vegetation generated by S3, the Great Wall ruins are extracted through the cloth simulation filtering algorithm to obtain the LiDAR point cloud data of the ruins; S5: Calculate the volume of the Great Wall ruins based on the LiDAR point cloud data of the ruins obtained in S4.

2. The method for measuring the volume of the Great Wall remains with irregular surface soil according to claim 1 is characterized in that: S1 includes: S11: Conduct environmental surveys to develop flight operation plans; S12: Design the flight route of the drone; S13: Deploy RTK base stations; S14: Flight verification, including test flights at the edge and center of the survey area to check the POS data quality and RTK fix rate; S15: Formal operations include conducting drone mapping of the survey area one by one in blocks or continuous flights according to the coverage of the RTK base station, real-time monitoring of communication status, and obtaining drone multispectral data and lidar data of the Great Wall ruins.

3. The method for measuring the volume of the Great Wall remains on irregular surface soil according to claim 1, characterized in that S2 include: S21: making a label image, including selecting hundreds or thousands of image sample data of the same size according to the image features of the vegetation in the DOM data, and extracting the label image of each sample data through interpretation; S22: training the model, including randomly selecting 2 / 3 of the prepared label images for deep learning model training through the Swin Transformer deep learning model; S23: Verify the model accuracy, including verifying the model accuracy through the remaining labeled images after the model training is completed.

4. The method for measuring the volume of the Great Wall remains on irregular surface soil according to claim 1, characterized in that S3 include: S31: pre-processing the laser radar point cloud data to obtain a point cloud area with vegetation; S32: Selecting initial seed points in the point cloud area with vegetation and constructing an initial ground model; S33: determining the remaining ground points in the point cloud area with vegetation and adding them to the initial ground model; S34: Verify the accuracy of the initial ground model and generate lidar point cloud data of the Great Wall ruins and surrounding environment without vegetation.

5. The method for measuring the volume of the Great Wall remains with irregular surface soil according to claim 4 is characterized in that: S32 includes: Divide the point cloud area with vegetation into grids, and select the lowest point of each grid as the initial seed point; Using the initial seed points, an initial ground model is constructed.

6. The method for measuring the volume of the Great Wall remains with irregular surface soil according to claim 5, characterized in that: S33 includes: The remaining points in the point cloud area with vegetation are sorted in ascending order of elevation, and whether to add to the initial ground model is determined point by point. The judgment condition is: if the point meets the following conditions, the point is marked as a ground point and added to the initial ground model; d i ≤h max And the i ≤θ max in: d i For node P i vertical distance to the nearest initial ground surface; h max is the maximum height difference threshold; θ i For node P i The angle with the normal vector of the initial ground; θ max is the maximum angle threshold.

7. The method for measuring the volume of the Great Wall remains on irregular surface soil according to claim 1, characterized in that S4 include: S41: Initialize the laser radar point cloud data of the Great Wall ruins and the surrounding environment without vegetation to form a cloth; S42: Calculate the gravity direction of the nodes in the cloth; S43: Calculate the gravity of the nodes in the cloth based on the gravity direction; S44: Calculate the spring forces of the nodes in the cloth; S45: Calculate the total force of the nodes in the cloth based on gravity and spring force; S46: Based on the total force, update the position of the nodes in the cloth mesh; S47: Collision detection, obtaining the latest vertical height of the node; S48: Repeat S44 to S47 until the cloth reaches a stable state or reaches a preset number of iterations; S49: Based on the updated position of the node, identify the Great Wall ruins entity.

8. A device for measuring the volume of the Great Wall remains on irregular surface soil, characterized in that: The device comprises: A data acquisition module (1) is used to acquire UAV multispectral data and LiDAR data of the Great Wall ruins, and pre-process the UAV multispectral data and LiDAR data to obtain LiDAR point cloud data and DOM data; A data extraction module (2) is used to extract the spatial range data of vegetation on and around the Great Wall based on the DOM data obtained by the data acquisition module (1) and in combination with an image recognition deep learning algorithm; A data removal module (3) is used to remove the vegetation point cloud in the laser radar point cloud data according to the vegetation spatial range data extracted by the data extraction module (2), so as to generate laser radar point cloud data of the Great Wall ruins and the surrounding environment without vegetation; The body extraction module (4) is used to extract the Great Wall ruins body based on the laser radar point cloud data of the Great Wall ruins and the surrounding environment without vegetation generated by the data removal module (3), and obtain the laser radar point cloud data of the ruins body by using a cloth simulation filtering algorithm; The volume calculation module (5) is used to calculate the volume of the Great Wall ruins based on the laser radar point cloud data of the ruins body obtained by the body extraction module (4).

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images