Method and device for measuring volume of irregular surface soil great wall remains, electronic equipment and computer storage medium

Abstract

Embodiments of the present application provide an irregular surface soil Great Wall remains volume measurement method, device, electronic equipment and computer storage medium. The method comprises: acquiring laser radar point cloud data and DOM data of the Great Wall site by a UAV; based on the DOM data, extracting vegetation spatial range data on the Great Wall body and the surrounding environment; generating laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation, and obtaining site body laser radar point cloud data therefrom, and calculating the volume of the Great Wall site. The method can effectively obtain the three-dimensional model and volume of the Great Wall remains in a large range, and realize the comparative monitoring of the surface morphology changes of the Great Wall in different periods, which can promote the fine development of the Great Wall protection research work.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of irregular surface metrology, in particular to a method and device for measuring the volume of an irregular surface Great Wall site, an electronic device and a computer storage medium. BACKGROUND

[0002] The Great Wall is the largest and most widely distributed linear cultural heritage in existence, with a presence in 404 counties across 15 provinces (regions and cities). In the process of investigation and protection, the quantitative description of the existing condition of the Great Wall, such as the volume of the remaining site, is often involved. Due to the long history and severe weathering of the existing wall, there are large differences in height, width, etc., and the surface is very irregular. Moreover, the terrain along the wall is complex and there are many trees and shrubs, making it difficult to accurately measure the volume. The Great Wall site in the northwest is the most typical, and there is an urgent need to design a method that can accurately measure the volume of the irregular surface Great Wall site in the northwest, so as to better study and protect the Great Wall site.

[0003] Since 2006, the State Administration of Cultural Heritage has carried out surveying and mapping of the Ming Great Wall and resource investigation, but has not involved three-dimensional modeling of the site itself. In recent years, the Ministry of Culture and Tourism Key Laboratory has carried out the construction of a comprehensive three-dimensional model database of the Ming Great Wall environment, but has not further carried out fine extraction of the wall. Some scholars generally limit the volume measurement of the Great Wall site to rough measurement and estimation at the outline level when carrying out surveying and mapping of important points and sections of the Great Wall. They choose areas with relatively simple environment and little vegetation interference, or choose Great Wall points and sections with relatively regular shape and easy to measure. With the rapid development of unmanned aerial vehicles and remote sensing sensors, the fine surveying and mapping and digitalization of the Great Wall have been technically supported, but the current application is limited to small-scale experiments. The technical system for general-purpose or large-scale investigation and modeling has not yet been formed. For example, the important branch technologies of vegetation interference removal and accurate modeling of remaining wall have not yet matured and need to be researched and supplemented. SUMMARY

[0004] Therefore, the present application aims to provide a method and device for measuring the volume of an irregular surface Great Wall site, an electronic device and a computer storage medium, which can effectively obtain a three-dimensional model and volume of the Great Wall site in a large area.

[0005] To achieve the above-mentioned purpose, the technical solutions adopted by the embodiments of the present application are as follows:

[0006] In a first aspect, the present application provides a method for measuring the volume of an irregular surface Great Wall site, comprising:

[0007] S1: obtaining unmanned aerial vehicle multispectral data and laser radar data of the Great Wall site, and preprocessing the unmanned aerial vehicle multispectral data and laser radar data to obtain laser radar point cloud data and DOM data;

[0008] S2: Based on the DOM data obtained in S1, the image recognition deep learning algorithm is combined to extract the spatial range data of the vegetation on and around the Great Wall body;

[0009] S3: According to the vegetation spatial range data extracted in S2, the vegetation part of the point cloud in the laser radar point cloud data is removed to generate laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation;

[0010] S4: Based on the laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation generated in S3, the Great Wall site body is extracted through the cloth simulation filtering algorithm to obtain the site body laser radar point cloud data;

[0011] S5: Based on the site body laser radar point cloud data obtained in S4, the volume of the Great Wall site is calculated.

[0012] In an optional embodiment, S1 comprises:

[0013] S11: Conduct an environmental survey to develop a flight operation plan;

[0014] S12: Design the flight route of the unmanned aerial vehicle;

[0015] S13: Layout RTK base station;

[0016] S14: Flight verification, including test flight at the edge and center of the survey area, checking the quality of POS data and RTK fixation rate;

[0017] S15: Formal operation, including block or continuous flight according to the coverage range of the RTK base station, unmanned aerial vehicle surveying and mapping of the survey area one by one, real-time monitoring of communication status, obtaining unmanned aerial vehicle multispectral data and laser radar data of the Great Wall site.

[0018] In an optional embodiment, S2 comprises:

[0019] S21: Making a label image, including selecting hundreds or thousands of image sample data with consistent size according to the image features of the vegetation in the DOM data, and extracting the label image of each sample data through interpretation;

[0020] S22: Training the model, including randomly selecting 2 / 3 of the prepared label images to train the deep learning model through the Swin Transformer deep learning model;

[0021] S23: Verify the model accuracy, including verifying the model accuracy through the remaining label images after the model training is completed.

[0022] In an optional embodiment, S3 comprises:

[0023] S31: preprocessing the laser radar point cloud data to obtain a point cloud area with vegetation;

[0024] S32: selecting an initial seed point in the point cloud area with vegetation and constructing an initial ground model;

[0025] S33: determining the remaining ground points in the point cloud area with vegetation and adding them to the initial ground model;

[0026] S34: verifying the accuracy of the initial ground model to generate laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation.

[0027] In an optional embodiment, S32 comprises:

[0028] dividing the point cloud area with vegetation into grids and selecting the lowest point of each grid as the initial seed point;

[0029] constructing the initial ground model using the initial seed point.

[0030] In an optional embodiment, S33 comprises:

[0031] sorting the remaining points in the point cloud area with vegetation in ascending order of elevation and determining whether to add them to the initial ground model point by point, wherein the determination condition is: if the point satisfies the following conditions, the point is marked as a ground point and added to the initial ground model.

[0032] d i ≤h max and θ i ≤θ max

[0033] wherein:

[0034] d i is the vertical distance from the node P i to the nearest initial ground;

[0035] h max is the maximum height difference threshold;

[0036] θ i is the angle between the normal vector of the node P i and the initial ground;

[0037] θ max is the maximum angle threshold.

[0038] In an optional embodiment, S4 comprises:

[0039] S41: initializing the laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation to form a cloth;

[0040] S42: Calculate the gravity direction of the nodes in the cloth;

[0041] S43: Calculate the gravity of the nodes in the cloth based on the gravity direction;

[0042] S44: Calculate the spring force of the nodes in the cloth;

[0043] S45: Calculate the total force of the nodes in the cloth based on the gravity and the spring force;

[0044] S46: Update the position of the nodes in the cloth based on the total force;

[0045] S47: Collision detection, get the latest ground height of the nodes;

[0046] S48: Repeat S44 to S47 until the cloth reaches a stable state or reaches a preset number of iterations;

[0047] S49: Based on the updated position of the nodes and the ground height, identify the Great Wall site body.

[0048] In a second aspect, the present application provides an irregular surface soil Great Wall site volume measuring device, the device comprising:

[0049] A data acquisition module is configured to acquire unmanned aerial vehicle multispectral data and laser radar data of the Great Wall site, and to preprocess the unmanned aerial vehicle multispectral data and the laser radar data to obtain laser radar point cloud data and DOM data.

[0050] A data extraction module is configured to extract vegetation spatial range data on and around the Great Wall wall body based on the DOM data acquired by the data acquisition module and in combination with an image recognition deep learning algorithm.

[0051] A data elimination module is configured to remove vegetation part point clouds in the laser radar point cloud data based on the vegetation spatial range data extracted by the data extraction module, and to generate laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation.

[0052] A body extraction module is configured to extract the Great Wall site body through a cloth simulation filtering algorithm based on the laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation generated by the data elimination module, and to obtain site body laser radar point cloud data.

[0053] A volume calculation module is configured to calculate the volume of the Great Wall site based on the site body laser radar point cloud data obtained by the body extraction module.

[0054] In a third aspect, the present application provides an electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of the method of any one of the preceding embodiments when executing the computer program.

[0055] In a fourth aspect, the present application provides a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the method of any one of the preceding embodiments.

[0056] The irregular surface soil Great Wall remains volume measurement method provided by the embodiment of the present application belongs to the urgent technical method in the current Great Wall site digitization protection work, can effectively obtain the three-dimensional model and volume of the Great Wall remains in a large range, and realizes the comparison monitoring of the surface form changes of the Great Wall in different periods, and can promote the fine development of the Great Wall protection research work.

[0057] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, and it should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation to the scope, and for the ordinary skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0059] Fig. 1 A flow chart of the irregular surface soil Great Wall remains volume measurement method provided by the embodiment of the present application is shown;

[0060] Fig. 2 A composition schematic diagram of the irregular surface soil Great Wall remains volume measurement device provided by the embodiment of the present application is shown;

[0061] Fig. 3 A composition schematic diagram of the electronic device provided by the embodiment of the present application is shown.

[0062] Label: 1-data acquisition module; 2-data extraction module; 3-data elimination module; 4-body extraction module; 5-volume calculation module. DETAILED DESCRIPTION

[0063] In order to make the objectives, technical solutions and advantages of the embodiments of the present application more clear, the following will combine the drawings in the embodiments of the present application to clearly and completely explain the technical solutions in the embodiments of the present application, obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the ordinary skilled in the art without creative labor are within the scope of protection of the present application.

[0064] The following will be combined Figs. 1 to 3This embodiment describes the method and apparatus for measuring the volume of the Great Wall remains on irregular surfaces.

[0065] like Fig. 1 As shown, this embodiment provides a method for measuring the volume of irregular surface earthen Great Wall remains (hereinafter referred to as the "method"). Based on UAV lidar 3D modeling and multispectral imaging technology, the method utilizes data from multiple types of UAV sensors to conduct 3D digital modeling of irregular surface earthen Great Wall remains. It also removes trees and shrubs from the Great Wall and surrounding environment within the model, enabling refined extraction and volume measurement of the irregular Great Wall remains. This method is suitable for comprehensive surveys and detailed investigations of irregular Great Wall remains, as well as for monitoring changes before and after construction using the acquired high-precision 3D model data. It is easy to promote and operate. The method specifically includes the following steps:

[0066] S1: Acquire UAV multispectral data and lidar data of the Great Wall ruins, and preprocess the UAV multispectral data and lidar data to obtain lidar point cloud data and DOM (Digital Orthophoto Map) data.

[0067] The lidar point cloud data and DOM data were acquired using drone aerial photography technology, and the specific process is as follows:

[0068] S11: Conduct environmental surveys to develop flight operation plans.

[0069] Before the flight operation, it is necessary to collect a full range of data, such as topographic maps and images, related to the shooting area, to understand the topography, climate conditions, airports, and important facilities of the shooting area, and to conduct on-site surveys of the operation area, conduct comprehensive analysis and research, determine the airspace conditions of the flight area and the adaptability of the equipment to the mission, and formulate a detailed flight operation plan.

[0070] S12: Design the flight path of the drone.

[0071] Flight routes should be rationally standardized according to the location and range of the monitoring units and flight quality requirements. The resolution of acquired DOM data should be better than 5cm, and the lidar point cloud density should be better than 500pts / m². 2 The width of the data should be no less than 50 meters on each side of the Great Wall (referring to the principle of a minimum protection area of ​​50 meters in the "Overall Protection Plan for the Great Wall" issued by the Ministry of Culture and Tourism and the State Administration of Cultural Heritage). In areas with significant topographic relief, flight routes should be set according to altitude levels (or a terrain-following flight strategy should be adopted) to ensure the consistency of the true spatial resolution of the data under different topographic conditions.

[0072] a. Retrieve DOM data.

[0073] According to the job safety requirements, data quality and sensor parameters, the flight height of the UAV is calculated, and the calculation formula is as follows:

[0074] H = (f x GSD) / a

[0075] In the formula:

[0076] H is the flight height, unit: m;

[0077] f is the focal length of the sensor lens, unit: mm;

[0078] GSD is the imaging resolution, unit: m;

[0079] a is the sensor pixel size or point cloud spacing, unit: mm.

[0080] b. Obtain the laser radar point cloud data.

[0081]

[0082] h: UAV flight height (m);

[0083] υ: UAV flight speed (m / s);

[0084] ρ: point cloud density (point / m2);

[0085] f: pulse frequency of laser radar (Hz, point / s), the number of laser points emitted by laser radar per second, usually tens of thousands to hundreds of thousands of points / s;

[0086] μ: scanning efficiency, the proportion of actual effective scanning points, usually 0.5-1;

[0087] θ: scanning field angle (FOV) of laser radar, usually 30°-360°.

[0088] S13: Set up RTK (full name: Real-Time Kinematic, Chinese name: Real-Time Kinematic) base station.

[0089] Before the UAV flight operation, in order to improve the surveying and mapping accuracy, it is necessary to set up a mobile RTK base station in the surveying and mapping area, and the setting process is as follows:

[0090] Pre-planning: Divide the survey area by 3-5 km intervals on satellite maps, plan feasible RTK base station locations based on DEM (Digital Elevation Model) data and basic road data, and ensure that the signal link between the UAV and the RTK base station is not blocked by terrain during the surveying and mapping process. Plan a RTK base station location for each survey area and control point concrete pouring for the location. Obtain high-precision coordinates (millimeter level) through static observation (≥2 hours).

[0091] RTK base station erection: Erection of RTK base station according to the scheme, completion of coordinate solution and signal test.

[0092] S14: Flight verification: Short-time test flight at the edge and center of the survey area to check the quality of POS (position and orientation system) data and RTK fixing rate.

[0093] S15: Formal operation: According to the coverage range of the RTK base station, divide or continuously fly, and survey the survey area one by one with the UAV, monitor the communication state in real time, and obtain the UAV multispectral data and laser radar data of the Great Wall site.

[0094] The step of preprocessing the UAV multispectral data and laser radar data includes:

[0095] Generating DOM data, including denoising, contrast enhancement, uniform light and color, geometric correction, radiation correction, image stitching, etc.

[0096] Generating laser radar point cloud data, including data analysis, point cloud noise removal, coordinate conversion and format conversion, point cloud segmentation and classification, motion distortion compensation, data compression and storage, etc.

[0097] In this way, when obtaining laser point cloud data, high-precision positioning RTK base stations are planned and arranged, which provides a solid guarantee for the surveying and mapping accuracy of the obtained point cloud data, and realizes the monitoring of the three-dimensional shape changes of the Great Wall site at the same place at different times based on laser radar point cloud data, supporting the fine monitoring and evaluation and protection of the Great Wall site.

[0098] S2: Based on the DOM data obtained in S1, combined with image recognition deep learning algorithm, extract the spatial range data of vegetation (such as trees and shrubs) on and around the Great Wall.

[0099] Specifically, vegetation filtering algorithms can be used to obtain digital elevation topography data with vegetation removed from lidar point cloud data. However, the basic principle of vegetation removal from lidar point cloud data is based on the elevation abrupt changes (local discontinuities) between neighboring lidar point clouds to determine vegetation points. Under normal natural terrain conditions, vegetation can be identified well, but complex terrain or many interfering factors can lead to large discrimination errors. For example, in this application, the Great Wall is a man-made structure with obvious elevation abrupt changes, and the identified vegetation is close to or on the Great Wall. In this case, the discrimination error of the vegetation filtering algorithm will increase significantly. Therefore, this application uses an image recognition deep learning algorithm to extract the spatial range data of vegetation, as follows:

[0100] S21: Create label images.

[0101] Based on the image features of vegetation in the DOM data, hundreds or thousands of image sample data of the same size (usually 512×512 pixels) are selected, and the label image of each sample data is extracted by interpretation.

[0102] S22: Training the model.

[0103] Using the Swing Transformer deep learning model, two-thirds of the pre-made label images were randomly selected for deep learning model training.

[0104] Among them, the Swin Transformer deep learning model is a visual Transformer model proposed by Microsoft Research Asia. It is designed for efficient processing of high-resolution images. Its core innovation lies in the hierarchical architecture and shifted window self-attention, which solves the bottleneck of computational complexity in traditional Transformer models. This makes it perform well in tasks such as image classification, object detection and semantic segmentation, and is especially suitable for fine extraction of vegetation in UAV remote sensing.

[0105] The core innovations of the Swin Transformer deep learning model are detailed below:

[0106] 1) Layered structure

[0107] Limitations of traditional Transformer models: ViT (Vision Transformer) segments images into fixed-size blocks (e.g., 16×16), but lacks the ability to represent multi-scale features.

[0108] Improvements to the Swin Transformer deep learning model: By merging image blocks layer by layer (PatchMerging), a pyramid structure similar to CNN (Convolutional Neural Networks) is constructed, forming 4 stages:

[0109] Stage 1: Input image is divided into 4x4 blocks, feature map size is:

[0110]

[0111] Stage 2-4: Merge adjacent blocks step by step, feature map size is reduced to:

[0112]

[0113] Supports multi-scale feature extraction, suitable for different granularity visual tasks (such as local texture and global distribution of vegetation).

[0114] 2) Shift window self-attention

[0115] Window division: Divide the feature map into non-overlapping local windows (such as 7x7 size), and calculate self-attention independently in each window.

[0116] Computational complexity optimization: The computational complexity within the window is O(M 2 ×N, M is the window size, N is the total number of image blocks), which is much lower than the global attention O(N 2 ).

[0117] Window shift: In adjacent layers, the window is shifted to the lower right corner by half the window size (such as from (0,0) to (3,3)), breaking the window boundary restrictions.

[0118] Effect: Enhance cross-window context interaction, avoid local information island.

[0119] Mathematical expression: Window self-attention formula:

[0120]

[0121] Where B is the relative position encoding, which enhances spatial perception.

[0122] S23: Verify model accuracy.

[0123] After the model training is completed, the model accuracy is verified through the remaining label images, and the verification indicators include accuracy, precision, recall, F1 score, and intersection over union.

[0124] a. Accuracy: represents the proportion of samples that the model predicts correctly, is one of the most intuitive and commonly used evaluation indicators, the formula is as follows:

[0125]

[0126] In the formula:

[0127] TP (True Positive): the number of samples correctly predicted by the model as positive (vegetation);

[0128] TN (True Negative): the number of samples correctly predicted by the model as negative (non-vegetation);

[0129] FP (False Positive): the number of samples incorrectly predicted by the model as positive;

[0130] FN (False Negative): the number of samples incorrectly predicted by the model as negative.

[0131] b. Precision: also known as the precision rate, represents the proportion of samples actually positive among those predicted as positive, the formula is as follows:

[0132]

[0133] In the formula:

[0134] TP: the number of correctly predicted vegetation pixels;

[0135] FP: the number of non-vegetation pixels incorrectly predicted as vegetation.

[0136] c. Recall: also known as the recall rate, represents the proportion of all actual positive samples correctly predicted as positive by the model, the formula is as follows:

[0137]

[0138] In the formula:

[0139] TP: the number of correctly predicted vegetation pixels;

[0140] FN: the number of true vegetation pixels not identified by the model (missed).

[0141] d. F1 Score: is the harmonic mean of precision and recall, trying to find a balance between the two indicators, especially suitable for handling unbalanced data sets, the formula is as follows:

[0142]

[0143] In the formula:

[0144] Precision: Precision;

[0145] Recall: Recall.

[0146] e. Intersection over Union (IoU): measures the degree of overlap between the prediction result and the true label, which is a very important evaluation index in semantic segmentation, and the formula is as follows:

[0147]

[0148] In the formula:

[0149] TP: the number of correctly predicted vegetation pixels;

[0150] FP: the number of non-vegetation pixels incorrectly predicted as vegetation;

[0151] FN: the number of real vegetation pixels that are not identified.

[0152] S24: Extracting vegetation spatial range data.

[0153] Using the trained model, the DOM data of the unmanned aerial vehicle is segmented to extract the vegetation spatial range data.

[0154] S3: According to the vegetation spatial range data extracted in S2, the vegetation part of the laser radar point cloud data is removed to generate laser radar point cloud data without vegetation for the Great Wall site and surrounding environment.

[0155] Specifically, S3 includes the following steps:

[0156] S31: Preprocessing the laser radar point cloud data to obtain the point cloud area with vegetation.

[0157] In ArcGIS Pro2.6 or later version software, the vegetation spatial range data extracted in S2 is used to crop the laser radar point cloud data to obtain the point cloud area with vegetation.

[0158] S32: Selecting an initial seed point in the point cloud area with vegetation to construct an initial ground model.

[0159] Divide the point cloud area with vegetation into a grid (the side length of the grid L, L is usually 2-3 times the average distance of the initial seed points), and select the lowest point of each grid as the initial seed point. The formula for generating the initial seed point is as follows:

[0160]

[0161] In which:

[0162] S0 is the initial seed point set, which is used to construct the initial triangular net (TIN);

[0163] P i is the point cloud of the vegetation area;

[0164] P v is the point cloud subset of the vegetation area;

[0165] Cell m is the first grid, and the study area is divided into square grids with a side length of L; m is the first grid, and the study area is divided into square grids with a side length of L;

[0166] z i is the elevation value (unit: meters) of point P i ;

[0167] Min is to take the point with the lowest elevation in the grid unit as the initial seed point.

[0168] Thus, the initial seed point of the initial selection can be generated using S0 to construct the initial ground model.

[0169] S33: Determine the remaining ground points in the point cloud area with vegetation and add them to the initial ground model.

[0170] Specifically, the remaining points in the point cloud area with vegetation are sorted in ascending order of elevation, and each point is judged whether to be added to the initial ground model. The judgment method of the remaining ground points is as follows:

[0171] 1) Calculate the vertical distance of the point to the initial ground (triangular face):

[0172]

[0173] Wherein:

[0174] d i is the vertical distance of point P i to the nearest initial ground;

[0175] x i , y i , z i are the three-dimensional coordinates of point P i ;

[0176] ax+by+cz+d=0 is the equation of the initial ground, and a, b, c, d are coefficients.

[0177] 2) Calculate the angle threshold between the point and the triangular face normal vector:

[0178]

[0179] Wherein:

[0180] θi is the normal vector of the initial ground surface; i is the angle between the normal vector of the initial ground surface and the local normal vector of the point P;

[0181] n t is the normal vector of the initial ground surface;

[0182] n i is the normal vector of the initial ground surface; i is the local normal vector of the point P.

[0183] 3) Judgment condition: if the following conditions are met, the point P i is marked as a ground point and added to the initial ground model.

[0184] d i ≤ h max and θ i ≤ θ max

[0185] wherein:

[0186] h max is the maximum height difference threshold (e.g. 0.5-2.0m, take a low value for flat areas and a high value for mountainous areas);

[0187] θ max is the maximum angle threshold (e.g. 5°-15°).

[0188] S34: Verify the accuracy of the initial ground model and generate laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation.

[0189] Specifically, the classification accuracy in the initial ground model is calculated using the ground truth point cloud.

[0190]

[0191] wherein:

[0192] TP (full name: True Positive) is the number of correctly classified ground points;

[0193] TN (full name: True Negative) is the number of correctly classified non-ground points (such as vegetation points);

[0194] FP (full name: False Positive) is the number of misclassified non-ground points as ground points;

[0195] FN (full name: False Negative) is the number of misclassified ground points as non-ground points.

[0196] In this way, in the case of meeting the accuracy (precision) standard, it is determined that the initial ground model contains the Great Wall site and the surrounding environment without vegetation laser radar point cloud data; in the case of not meeting the accuracy (precision) standard, S3 is re-executed.

[0197] In S2 and S3, in order to avoid the interference of trees and shrubs on the Great Wall wall or the surrounding environment on the three-dimensional modeling of the Great Wall site, the spatial range data of the trees and shrubs are accurately extracted through the image recognition deep learning algorithm, and the three-dimensional point cloud data of the Great Wall site and the environment not affected by the tall vegetation are further calculated and obtained, compared with directly through the vegetation filtering algorithm, the problem that the vegetation and wall point cloud are difficult to distinguish due to the common height mutation of the wall and tall vegetation relative to the natural ground surface is effectively avoided.

[0198] S4: Based on the laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation generated in S3, the Great Wall site body extraction is performed through the cloth simulation filtering algorithm, and the site body laser radar point cloud data is obtained.

[0199] Specifically, S4 includes the following steps:

[0200] S41: Initialize the laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation to form cloth.

[0201] The point cloud data is regarded as a surface, and is flipped, a layer of cloth is simulated to cover on the flipped surface, and the initial cloth grid is formed, the grid size is 1 to 2 times the average distance of the point cloud, and the initial position of the "cloth" is above the highest point of the initial ground.

[0202] S42: Calculate the gravity direction of the nodes in the cloth.

[0203] Determine a gravity direction vector:

[0204] S43: Calculate the gravity of the nodes in the cloth based on the gravity direction.

[0205] For each node P on the cloth i , calculate the gravity it receives:

[0206]

[0207] Wherein, m is the mass of the node.

[0208] S44: Calculate the spring force of the nodes in the cloth.

[0209] For each node P on the cloth i , calculate the spring force between it and the adjacent node P j . The spring force F spring,ij can be calculated by Hooke's law:

[0210]

[0211] Where k is the spring constant. It is the elongation of the spring, i.e., P. i and P j The difference between the distance between them and their initial distance.

[0212] S45: Calculate the total force at the nodes in the fabric based on gravity and spring force.

[0213] For each node P on the cloth i Calculate the total force F acting on it. total This includes gravity and spring force:

[0214]

[0215] In the formula, neighbors(P) i ) is node P i The set of adjacent nodes.

[0216] S46: Update the position of nodes in the cloth mesh based on the total force.

[0217] According to Newton's second law a i =F total / m, calculate the acceleration a at each node. i Then update the node's position:

[0218]

[0219] in, Let be the velocity at time t, and Δt be the time step. Let t be the position of the node at time t.

[0220] S47: Collision detection, determining the ground height of the node.

[0221] For each grid node P i Search for it The nearest point cloud qk = (x) below the vertical projection of the position k y k , z k If z k ≥h i Node P i If "blocked", then record z. k For node P i The corresponding ground height, where h i For node P i The latest vertical height.

[0222] S48: Repeat S44 to S47 until the cloth reaches a steady state or reaches a preset number of iterations.

[0223] S49: Based on the updated position of the node and the ground height, the Great Wall site body is identified.

[0224] After the cloth reaches a steady state or reaches a preset number of iterations, the height of the nodes in the cloth that is higher than or equal to the surrounding point cloud surface is identified, and these nodes are the parts of the Great Wall site body.

[0225] The setting of cloth simulation algorithm parameters directly affects the filtering effect. The method defines parameters mainly including point cloud data grid resolution and spring coefficient: 1) The grid resolution represents the size of the grid of the simulated cloth. The larger the parameter setting, the rougher the simulated terrain, and vice versa. Therefore, for different terrain undulations, different grid sizes can be set; 2) The spring coefficient is the hardness of the cloth. The larger the value, the tighter the cloth. By adjusting the coefficient size, different terrain conditions can be adapted.

[0226] Through the above calculation, the laser radar point cloud data of the Great Wall site body is obtained.

[0227] In S4, a cloth filtering algorithm commonly used to extract vegetation is used for Great Wall body information extraction, which belongs to the application of technology transfer innovation. This method can adapt to the extraction of Great Wall body laser radar point cloud information under various terrain conditions by adjusting parameters. The method has good universality and is more suitable for Great Wall site information extraction in the diversified terrain background of the northwest region.

[0228] S5: Based on the site body laser radar point cloud data obtained in S4, the volume of the Great Wall site is calculated.

[0229] Specifically, S5 includes the following steps:

[0230] S51: Triangulate the laser radar point cloud data to convert discrete point cloud data into a continuous geometric body and construct a three-dimensional surface model composed of multiple triangles.

[0231] S52: Decompose the three-dimensional surface model into small tetrahedrons and calculate the volume of each tetrahedron.

[0232] The three-dimensional surface model is decomposed into small tetrahedrons, and the total volume is obtained by calculating the volume of each tetrahedron and summing it up. The volume of a tetrahedron can be calculated by the coordinates of its four vertices, and the formula is:

[0233]

[0234] where, is the vector from one vertex of the tetrahedron to the other three, · denotes the dot product, and x denotes the cross product.

[0235] S53: Sum the volumes of all tetrahedrons to obtain the total volume of the entire three-dimensional model:

[0236]

[0237] In the formula: V is the volume of the Great Wall site, v i is the volume of the ith tetrahedron.

[0238] The volume of the Great Wall site body is calculated by the above method.

[0239] The irregular surface soil Great Wall remains volume measurement method provided by the embodiment has the following beneficial effects:

[0240] 1. The method belongs to the urgent technical method required in the current digital protection work of the Great Wall site, can effectively obtain the three-dimensional model and volume of the Great Wall remains in a large range, and realizes the comparison and monitoring of the surface morphological changes of the Great Wall in different periods, which can promote the fine development of the Great Wall protection research work;

[0241] 2. The unmanned aerial vehicle multispectral and laser radar basic equipment required by the method is mature in technology, which provides a basic technical guarantee for the feasibility of the method;

[0242] 3. The technical process of the method is easy to master, convenient to operate, high in accuracy, and strong in universality, which can not only serve the soil Great Wall remains investigation in scientific research work, but also provide a set of large-scale northwest soil Great Wall remains volume survey technical method for cultural relic protection.

[0243] The irregular surface soil Great Wall remains volume measurement device provided by the embodiment is described below, and the irregular surface soil Great Wall remains volume measurement device described below can be correspondingly referred to the irregular surface soil Great Wall remains volume measurement method described above.

[0244] As Fig. 2 shown, the irregular surface soil Great Wall remains volume measurement device includes a data acquisition module 1, a data extraction module 2, a data elimination module 3, a body extraction module 4, and a volume calculation module 5.

[0245] The data acquisition module 1 is used to acquire unmanned aerial vehicle multispectral data and laser radar data of the Great Wall site, and to pre-process the unmanned aerial vehicle multispectral data and laser radar data to obtain laser radar point cloud data and DOM data;

[0246] The data extraction module 2 is used to extract the vegetation space range data on and around the Great Wall wall body based on the DOM data acquired by the data acquisition module 1, in combination with an image recognition deep learning algorithm;

[0247] The data elimination module 3 is configured to eliminate the vegetation part of the point cloud in the laser radar point cloud data according to the vegetation spatial range data extracted by the data extraction module 2, and generate laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation.

[0248] The body extraction module 4 is configured to extract the body of the Great Wall site by a cloth simulation filtering algorithm based on the laser radar point cloud data of the Great Wall site and the surrounding environment without vegetation generated by the data elimination module 3, and obtain the site body laser radar point cloud data.

[0249] The volume calculation module 5 is configured to calculate the volume of the Great Wall site based on the site body laser radar point cloud data obtained by the body extraction module 4.

[0250] Fig. 3 An example of a schematic diagram of the physical structure of an electronic device is shown, which can be a smart terminal, and the internal structure diagram can be as shown in Fig. 3 The electronic device includes a processor, a memory, and a network interface connected by a system bus. The processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the electronic device is configured to communicate with external terminals through network connection. The computer program is executed by the processor to implement the above-mentioned irregular surface soil Great Wall site volume measurement method.

[0251] Those skilled in the art can understand that Fig. 3 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0252] On the other hand, the embodiment also provides a computer storage medium storing a computer program, and the computer program is executed by a processor to implement the above-mentioned irregular surface soil Great Wall site volume measurement method.

[0253] In yet another aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer readable storage medium, and the processor implements the above-mentioned irregular surface soil Great Wall site volume measurement method when executing the computer instructions.

[0254] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory.

[0255] By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0256] The technical features of the above embodiments can be combined in any way. In order to make the description simple, not all possible combinations of the technical features in the above embodiments are described, but as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0257] The above embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of the present application. Therefore, the scope of protection of the patent of the present application should be subject to the appended claims.

Claims

1. A method for measuring the volume of irregular surface earthen Great Wall remains, characterized in that, The method includes: S1: Acquire UAV multispectral data and lidar data of the Great Wall ruins, and preprocess the UAV multispectral data and lidar data to obtain lidar point cloud data and DOM data; S2: Based on the DOM data obtained from S1, combined with image recognition deep learning algorithms, extract the spatial range data of vegetation on and around the Great Wall. S3: Based on the vegetation spatial range data extracted in S2, remove the vegetation portion of the point cloud data from the lidar point cloud data to generate lidar point cloud data of the Great Wall ruins and surrounding environment without vegetation. S4: Based on the lidar point cloud data of the Great Wall ruins and surrounding environment generated in S3 without vegetation, the Great Wall ruins are extracted using a cloth simulation filtering algorithm to obtain lidar point cloud data of the ruins. S5: Calculate the volume of the Great Wall ruins based on the lidar point cloud data of the ruins obtained in S4.

2. The method for measuring the volume of irregular surface earthen Great Wall remains according to claim 1, characterized in that, S1 includes: S11: Conduct environmental surveys to develop flight operation plans; S12: Design the flight path of the drone; S13: Deploy RTK base stations; S14: Flight validation, including test flights at the edge and center of the test area to check POS data quality and RTK fixation rate; S15: Formal operations, including flying in blocks or continuously according to the coverage of RTK base stations, conducting UAV mapping of the survey area one by one, monitoring the communication status in real time, and acquiring UAV multispectral data and lidar data of the Great Wall ruins.

3. The method for measuring the volume of irregular surface earthen Great Wall remains according to claim 1, characterized in that, S2 include: S21: Create labeled images, including selecting hundreds or thousands of image sample data of the same size based on the image features of vegetation in the DOM data, and extracting the labeled images of each sample data through interpretation; S22: Training the model, including training the deep learning model by randomly selecting 2 / 3 of the prepared label images using the Swing Transformer deep learning model; S23: Verify model accuracy, including verifying model accuracy using the remaining labeled images after model training is completed.

4. The method for measuring the volume of irregular surface earthen Great Wall remains according to claim 1, characterized in that, S3 include: S31: Preprocess the lidar point cloud data to obtain point cloud areas with vegetation; S32: Select initial seed points in a point cloud region with vegetation to construct an initial ground model; S33: Identify the remaining ground points in the point cloud region with vegetation and add them to the initial ground model; S34: Verify the accuracy of the initial ground model and generate lidar point cloud data of the Great Wall ruins and surrounding environment without vegetation.

5. The method for measuring the volume of irregular surface earthen Great Wall remains according to claim 4, characterized in that, S32 includes: The point cloud region with vegetation is divided into grids, and the lowest point of each grid is selected as the initial seed point. Use the initial seed points to construct the initial ground model.

6. The method for measuring the volume of irregular surface earthen Great Wall remains according to claim 5, characterized in that, S33 includes: For the remaining points in the point cloud region with vegetation, sort them in ascending order of elevation, and determine whether to add them to the initial ground model for each point. The determination condition is: if a point meets the following conditions, then the point is marked as a ground point and added to the initial ground model. d i ≤h max And the i ≤θ max in: d i For node P i The vertical distance to the nearest initial ground surface; h max The maximum height difference threshold; θ i For node P i The angle between the ground and the normal vector of the initial ground; θ max This is the maximum angle threshold.

7. The method for measuring the volume of irregular surface earthen Great Wall remains according to claim 1, characterized in that, S4 include: S41: Initialize lidar point cloud data of the Great Wall ruins and surrounding environment without vegetation to form a cloth; S42: Calculate the direction of gravity at the nodes in the fabric; S43: Calculate the gravity of the nodes in the fabric based on the direction of gravity; S44: Calculate the spring force at the nodes in the fabric; S45: Calculate the total force at the nodes in the fabric based on gravity and spring force; S46: Update the position of nodes in the cloth mesh based on the total force; S47: Collision detection, determining the latest vertical height of the node; S48: Repeat S44 to S47 until the fabric reaches a stable state or the preset number of iterations is reached; S49: Identify the Great Wall ruins based on the updated node positions.

8. A device for measuring the volume of irregular surface earthen Great Wall remains, characterized in that, The device includes: The data acquisition module (1) is used to acquire UAV multispectral data and lidar data of the Great Wall site, and to preprocess the UAV multispectral data and lidar data to obtain lidar point cloud data and DOM data. The data extraction module (2) is used to extract the vegetation spatial range data on and around the Great Wall based on the DOM data obtained by the data acquisition module (1) and combined with the image recognition deep learning algorithm. The data removal module (3) is used to remove the vegetation part of the point cloud in the lidar point cloud data based on the vegetation spatial range data extracted by the data extraction module (2), and generate lidar point cloud data of the Great Wall ruins and surrounding environment without vegetation. The ontology extraction module (4) is used to extract the ontology of the Great Wall site and its surrounding environment without vegetation based on the lidar point cloud data generated by the data removal module (3) through the cloth simulation filtering algorithm, and obtain the ontology lidar point cloud data of the site. The volume calculation module (5) is used to calculate the volume of the Great Wall site based on the laser radar point cloud data of the site obtained by the ontology extraction module (4).

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images